Software Testing: Reinventing for Continuous Innovation

Software Testing - Contextual Background:

In many fields, testing is a common activity. Whether for hardware products such as small household devices (kitchen robots and coffee makers) or for more complex systems (such as airplanes, rockets, and nuclear power plants), all go through trials at different stages of their development before being put on the market. Software applications, which are ubiquitous in our daily lives, also follow this rule. The rise of connected devices and the increasing integration of software in these systems make testing more important than ever.

For all IT projects, such as developing a new product or evolving an IT system, the testing activity follows common principles: it generally involves stimulating the system/product under test, observing its behavior and/or properties, and involves a comparison with an ideal reference system. The goal is to detect errors and validate functionalities.

It thus becomes evident that test management represents a major challenge for organizations, directly impacting project success and delivery quality. Establishing a quality program based on good testing practices, including automation, is an integral part of this approach.

Here is a definition of software testing according to the “American Society for Quality” (ASQ):
“Testing is the execution or evaluation of a system or component, by automated or manual means, to verify that it meets its specifications or to identify differences between expected results and actual results.”

Why Software Testing?

We conduct tests because we don’t always do our job right the first time! On average, in the industry, about 40% of the development budget is allocated to performing checks (redoing work). One of the reasons for having a quality group is to establish a software quality assurance (SQA) process to reduce correction time (testing time). Having a dedicated testing team compels the project team to better develop their product, as they are no longer solely the verifier and responsible for testing. The quality group supports project teams and the testing team to ensure compliance with the established quality plan. Additionally, it evolves the testing program and the quality toolkit (processes, templates, tools, guides, etc.) and ensures that continuous improvement (retrospective) is carried out adequately to benefit all teams within the organization (scaling).

Independence of Teams:

The organizational structure should aim for distinct division for good governance, including:
- Quality management in the organization
- Test management
- Management of IT project execution

Why Three Distinct Groups?

This model ensures independence between the "executor" and the "verifier," just as a quality office is independent of the project team. This division ensures better management of testing throughout the entire software lifecycle.

In summary, here’s the interaction between the three (3) groups:

• The project team builds a product according to a methodology.
• The testing team verifies the product and identifies weaknesses.
• The quality team controls and improves practices within the organization.

Summary of Roles and Responsibilities

The testing team and the quality assurance team serve the project team and have their own management and budget.

Project Team:

• Plan the execution of the product (schedule, costs, resources, etc.)
• Monitor the progress of product release development
• Develop the product (analysis, design, programming)
• Execute unit tests
• Perform deployments and production releases

Testing Team:

• Plan, design, execute, observe (result analysis), and track corrections (defect management) for integrated and system tests
• Automate test cases
• Produce evidence of unit tests
• Support acceptance testing
• Generate test metrics

Quality Assurance Team:

• Plan quality assurance (QA) and quality control (QC) activities
• Implement the quality program (processes, procedures, standards, etc.)
• Conduct quality control activities
• Report non-conformities
• Collect and produce quality metrics
• Train resources on applicable methods
• Communicate quality results
• Implement continuous improvement in practices
• Manage archives and risks based on observed non-conformities in different product versions

Benefits

Financial Benefits

The reorganization of groups allows for substantial savings during project execution.

• An investment of 6% of project efforts in software quality assurance (SQA) with the implementation of good testing practices by the testing team can reduce testing efforts by at least 50% and project efforts by 30%. (Source: American Society for Quality)

NOTE: The speed of transition depends on **management involvement** and the seriousness of the quality program.

• Quality is free after a certain time if there are improvement loops. Typically, three (3) loops are needed before monetary benefits are realized for the organization. However, increased quality can be observed as early as the first loop.

NOTE: A loop means:

• A project or evolution has passed through the entire development process applying quality assurance and control at each stage.
• The testing team participated from the beginning of the project and conducted its tests following good testing practices.
• Quality assurance provides corrective measures to the development process and the testing process based on post-mortem results and documented anomalies from the testing team.

Benefits During Review Meetings

One of the advantages of separating the testing team from the project team is obtaining two (2) different perspectives on project progress.
A project manager may inform management that the project is under control, that costs and time are respected, but what about quality?
- Did they cut corners on quality to achieve their ends?
- Are testing phases being overlooked to meet deadlines and budgets?
- Did they deliver an acceptable level of quality to the testing team?
- Were unit tests performed?
- A project can be "red" in quality but "green" in management, and vice versa.

Conclusion

Software quality assurance (SQA) brings maturity to the organization by innovating, standardizing processes, tools, and practices. The dedicated testing team complements your development methodologies, regardless of your approach (Agile, DevOps, Hybrid, or even traditional).

Good quality management and a testing team are strategic assets for any organization seeking to improve product quality, optimize processes, and remain competitive in the market. Centralization of knowledge, innovation, and continuous improvement of processes are the pillars of this approach.

To go further: