Fundamental Principles In Software Testing - 29 Key Points

The goal of testing is to find defects before customers find them out.
Exhaustive testing is not possible; program testing can only show the presence of defects, never their absence.
Testing applies all through the software life cycle and is not an end-of-cycle activity.
Understand the reason behind the test.
Test the tests first.
Tests develop immunity and have to be revised constantly.
Defects occur in convoys or clusters, and testing should focus on these convoys.
Testing encompasses defect prevention.
Testing is a fine balance of defect prevention and defect detection.
Intelligent and well-planned automation is key to realizing the benefits of testing.
Testing requires talented, committed people who believe in themselves and work in teams.
The cost of building a product and the number of defects in it increase steeply with the number of defects allowed to seep into the later phases of software development.
Costs due to defects in software have a compounding effect. The cost to fix them increases exponentially as they progress from requirements phase, through the design phase, coding phase, testing phase and post release phase. The factor can be in thousands. This makes identifying defects in an early phase crucial for a product's profitable operations.
A defective test is more dangerous than a defective product.
We can choose to execute only a subset of tests that cover a maximum number of possible errors. This is due to the limited amount of time to test the product. We can never be 100% sure that there are no defects left out.
Smaller the lag time between defect injection and defect detection, lesser are the unnecessary costs. Thus, it becomes essential to catch the defects as early as possible.
A defect introduced during the requirements phase that makes it to the final release may cost as much as a thousand times the cost of detecting and correcting the defect during requirements gathering itself.
Testing requires asking about and understanding what you are trying to test, know what the correct outcome is, and why you are performing any test. If we carry out tests without understanding why we are running them, we'll end up running inappropriate tests that do not address what the product should do.
It may even turn out that the product has been modified to make sure that tests are run successfully, even if the product doesn't meet the intended purpose. This is a recipe for disaster after the final release.
Defects develop immunity against test cases. As and when we write new test cases and uncover new defects in the product, other defects that were 'hiding' underneath show up. Tests have to be constantly revised to tackle new defects.
The probability of existence of more errors in a section of program is proportional to the number of errors already found in that section. The number of defects yet to be found increases with the number of defects uncovered.
Testing can only find a part of defects that exist in a cluster. Fixing a defect may introduce another defect to the cluster. A fix for one defect generally introduces some instability and necessitates another fix. All these fixes produce side effects that eventually cause a convoy of defects in certain parts of product. The long term solution in such a case is to re-architecture the design and rewrite the code.
Testers have to work with development engineers to make sure that the root cause of the defects are addressed. Defect prevention is a part of tester's job. They have to strike a balance between defect prevention and defect detection.
Defect detection and correction is called Quality Control, while defect prevention is called Quality Assurance. These are two methods to achieve quality. Testing is traditionally considered a Quality control activity. Quality assurance is associated with process models like CMM, CMMI, ISO 9001 etc.
These two function should not be viewed as mutually exclusive, 'either-or' choices. Often organisations swing from one extreme to another, one rooting for defect prevention(quality assurance) and other rooting for defect detection(quality control). This should be avoided. Quality control and assurance are supplementary activities, and should be done in a right mix to develop a quality product. This grid describes the relationship between quality control and quality assurance.
Defect prevention is process focussed and defect detection is product focused. Defect detection acts as an extra check to augment the effectiveness of defect prevention.
Quality is institutionalized with consistently high focus on both defect prevention and detection. Although, an organisation may have to allocate sufficient resources for sustaining a high level of both Quality control and quality assurance activities.
The relative emphasis placed on the defect detection and prevention will vary on the type of product, closeness of release date and resources available. Making a conscious choice of balance will enable an organisation to produce better quality products. It is important not to over-emphasize one of these at the expense of the other.
Automation in Software Testing : Some Points to Consider
- Automated testing requires careful planning, evaluation and training. Automation may not produce immediate results.
- Expecting immediate results from automation can bring dissapointment and lead people to blame automation, instead of objectively looking at their level of preparedness for automation in terms of planning, evaluation and training.
- A failure in doing so leads to organisations reverting back to manual testing. It is like a double edged sword.
- Know first why you want to automate and what you want to automate, before recommending automation just for automation's sake.
- Evaluate multiple tools before choosing one as being most appropriate for you needs.
- Try to choose tools to match your needs, rather than changing your needs to match the tool's capabilities.
- Train people first before expecting them to be productive.
- Do not expect overnight returns from automation.