Kaizen Technologies Inc.

Number of people on the test team: 10

Number of developers supported by the test team: 26

Best Practice Description:

INTRODUCTION : “Eagle Eye” is one of the architecture neutral software product developed by Kaizen Technologies Inc., USA . The product aims at achieving a monitoring system for heterogeneous application servers with web-based dashboard.

ABSTRACT : With enterprise applications playing a vital role in mission-critical business processes, managing their performance and availability has never been more important. Failure of key applications and services, even for a short time, can be devastating in terms of cost, market position, and long-term business viability. Complicating matters, most organizations must contend with growing complexity and constant change, making failure much more likely. The rigors of competition, regulatory compliance, and a dynamic marketplace promise that these trends will continue to escalate along with the need to reduce costs and lower risk. Eagle Eye uses the proven technological glue to bind together new and legacy applications and deliver greater responsiveness to users. To get the most out of these integrated applications users also need a unified console. Enterprise Portals provide significant out-of-the-box functionality for managing interrelated applications while giving users a highly customizable interface.

MAIN FEATURES: Real-time Performance and Transaction Monitoring, Continuous Memory Monitoring, Role-based Reporting and Visualization, Controller, Alarms and Notifications.

TYPE OF DEVELOPMENT : It is a green field product development and perfectly adheres to the product life cycle methodology.

PROJECT MANAGEMENT : A program manager from the project management office is responsible for the overall project management. The product design manager, development manger and the test manager will be interacting with him on a regular basis for the responsible deliverables. A well defined PMBOK methodology was put in place for the whole Software Development Life Cycle and Software Testing Life Cycle phase.

TEAM STRUCTURE : The team is structured as per the function set. The design, development and test team's were aligned to the work package classification based on the key functionalities in order to leverage the team work, functional understanding and improve their delivery capability.

CURRENT PHASE OF THE PRODUCT : Alpha Testing Phase.

OVERALL TEAM SIZE : 42 includes (PMO team-1, Design team-5, Development team-26 and Test team-10) TEST TEAM ROLES: Test Manager 1, Test Module Lead – 3, Testers – 6. The Test team is structured as every 2 set of Testers will report in to the Module Test lead and the module Test lead report to the Test manager. The Module test lead is a senior tester and allocated based on the product work package.

TESTING BEST PRACTICES :

• The test team is involved right from the kick off meeting. (To make the testing team to understand the project and test objective, timelines, quality criteria's etc.)
• Based on the testing expectation set, a well aligned Test Strategy and Test Plan was arrived. (Stake holder approved test strategy and a test plan which details the subtasks which the resources were aligned)
• Mandate was set to organize the test team to expedite the best testing practice in place with our previous testing experience and standard industry models.
• Identifying the test team members. (As decided to organize the team, based on the function sets; the inputs based on the resource competencies and the past projects they performed, the right set of test team members were identified)
• Training them and enabling them. (Build installation and deployment, QA environment Admin Roles, Test Harness etc.)
• The resource utilization excel was prepared and productivity guidelines were well established and linked. Resource utilization, Productivity, Defect leakages were linked to the KPI's as well linked to Annual and mid-term Appraisals.
• Communication and Escalation policies in place. (An approved and circulated communication plan and escalation policies were put in place to avoid decision / communication delays)
• Testing team involved right from the Requirement Analysis and Design handover phase. This has given a good understanding, so that the test scripts quality is really improved.
• Unit testing from development testing was in place. They have a unit testing exit criteria and upon meeting the criteria they are officially allowed to release the product to the next phase.
• Stakeholders approved test governance. Overall test governance was discussed with all the stakeholders, drafted, accepted, signed off and put in place for compliance.
• Quality Gates were introduced. (Test phase wise - entry and exit criteria set, agreed, signed off and circulated)
• Compliance to the agreed process (Agreed as well to the company's internal ISO / CMMi compliance)
• Assurance planned and built cumulatively. (As this is a green field product the assurance is planned iteratively for build to build and as well phase wise)
• Severity guidelines defined. (To avoid mail trails and long discussions during logging a defect, between the development and testing team involving the product management experts to define it from the criticality view of a business function perspective)
• Front loading of Business critical test cases testing to have early visibility. (It has been planned to execute the Business critical scripts upfront after a point of build stability during the Alpha testing phase itself to get a comfort of the functionalities developed and as well to identify the business critical scripts failure upfront)
• Rigorous regression testing process. (During the case of a Bug fix build testing the test team tests the bug fixes and also they will ensure the in and around functionalities are tested around by referring to the impact analysis and the traceability matrix document. Also the regression defects are marked as per the process in the defect management system so that the metrics will help us to analyse and act upon)
• Test Environment management. (The planned set of QA environments helped us to manage the builds which includes the separate bug fix test environment to test the hot fixes / patches before promoting them to the main test environment, helped us a lot in improving productivity)
• Collaborative approach. (The work package wise team allocation helped to have the overall understanding, but clear with the individual objectives set for them. A free flow of who is the functional author, who is the developer and who is the tester is known to all. The triage team were also based on the work package wise distribution)
• Tracking against the test plan and effort to complete. (The work package level, task level and the build level tracking against the plan and also the effort to complete was a complete end to end tracking as per the schedule)
• Test execution as per strategy. (The test execution phase was perfectly as per the approved test strategy)
• Enhanced test coverage. (Early expected key inputs from domain/business analysts received on test coverage. The high level test conditions/ summary, the test scripts and the test cases were tied up to the each requirement/ functionality so that even a single piece is not missed out. We have prepared a test type mapping excel and tied the test scripts to the test types to ensure that all the minimum/ mandatory testing types should have been tested during the test phase)
• Strengthened internal and external test script review process. The High level test Summary (i.e. High level test condition), Test plan and Test script preparation were self reviewed, peer reviewed and product management reviewed and signed off before baseline.
• Effective tools utilization (Defect management tool, Code coverage tools for the unit testing done by the development during the development unit testing phase, Requirement traceability, Test scripts repository and Version control, Bug/ Defect Life cycle, Change request life cycle etc.,)
• Early defect detection. (The key in this is to identify the key business critical issues upfront during the early phases. The baselined test scripts shared to the development team were executed before the build release to ensure issues were identified upfront. Also the business process test script was tested during the Alpha test phase to identify the business critical issues/ bugs/ defects upfront.)
• Root cause analysis.(Root casual analysis were done on the high order severity issues so as to find out the root cause to which phase this could have been avoided as well occurs from which build onwards. These data's will help us to take quick turn around decisions and move faster towards delivery)
• Test Execution and Continuous testing. (The test team will continue testing even though they finish the current phase of testing when the product is in the next phase, to ensure the enhancement / change request is not impacting the whole product quality, as it is proved that testing is not one time it should be continuous to yield more)
• Metrics collection, analysis and reporting for continuous improvement. (e.g. Phase wise Test Cases count and Size, Unit test issues, Severity wise issues, Total Open Issues at Phase Exit, Average Issues per function set, Regression Trend, Test phase wise Issue Distribution, High Severity Issue Trend, Re-Open trend etc to quote a few)
• Triage teams and effective Triage meeting were in place.
• Key functionalities thoroughly tested, Lessons learned considered
• A Dashboard was followed to share the Test results, Metrics and tracking the testing deliverables.

BOTTOMLINE : Improved testing productivity, Measurable test coverage, Early visibility of defects, Reduced SDLC cost, Improved quality and Reduced time to market.