Introduction

Like every process Software Testing also needs some benchmarks to determine if the Testing is progressing in the right direction. Software Testing Principles are the benchmarks on the basis of which we decide if the Testing approach being followed is correct. The Software Testing Principles ensures that optimum testing results are achieved without deviating from the core objective.

In a nutshell, Software Testing Principles help the testing team to function in a productive manner by making maximum utilization of time and efforts.

There are 7 software testing best practices:

1. Exhaustive testing is not possible

Exhaustive Testing as the name suggests is a time taking testing process, where the Testing team would target to test each and every permutation and combinations. This would require a lot of time and effort. Ironically, it’s very rare to accommodate such testing within the Project Timelines. Hence an effective way is to analyse the risk, prioritize the testing based on the risk and make use of testing techniques to get optimized results.

For Example: Imagine we have to test a page where there are 5 fields that accept alphabets, numbers or special characters and a numeric field that accepts a value from 1 lac to 10 lac. 

Now, if we perform exhaustive testing, we will have to enter all the combinations one by one and check for all the fields. Covering all the scenarios will take a lot of time. On the contrary, if we use a technique like ‘Equivalence Partitioning’ the testing would be completed in half time

2. Defect Clustering

It usually happens that in a complete feature there are only a few functionalities that will have the maximum no. of bugs. If we go by Pareto’s law 20% of the functionality has 80% of bugs. This is true for large applications that have multiple modules of different complexity. 

Read: Test Case Design Techniques with Examples

For Example: If we have a website that has 3 screens. The first is the signup/login page, the second screen is a form of competitive exam and the 3rd page is a payment screen that takes the card details and processes the payment. Now if asked to a Tester about what will be the most critical functionality of this website. The answer would surely be the Payment Screen. Which is very obvious. And so most of the bugs he would encounter will be around the payment process.

Similarly, shortlisting the modules and preparing the regression suite based on this principle helps identify maximum bugs in very less time. However, this would result in Pesticide Paradox if we do not constantly update the regression suite. We will now understand what Pesticide Paradox is.

3. Pesticide Paradox

If we use the same type of pesticide for a long period of time to kill insects, the insects gradually develop resistance against these pesticides. And it is no more effective on the insects. The same happens in Software testing as well. If we execute the same set of test cases, again and again, we would not be able to discover new defects in the functionality. This happens as the coverage becomes restricted and all the bugs logged around that area will already be fixed as the predefined scope has been tested multiple times.

This can be avoided if we keep revising the test cases. Adding new and different cases each time regression is required would help overcome this issue.

For Example: If a page needs to be tested having 5 fields 2 dropdowns, 2 textbox and 1 date field. Our dropdown fields depend on each other and hence mostly bugs are reported around these fields. Looking at the past observations the testing team has added test cases to verify the dropdown fields and left the other 3 fields. Now there can be chances when the dropdown works fine but the date field is not accepting correct values.

4. Testing shows the presence of defects

Testing is done with the belief that the application has bugs. Even when the testers are not finding any bugs does not mean that the application is 100 % defect-free. Therefore, it is important to design test cases with maximum coverage.

5. Absence of Error - Fallacy

This principle focuses on checking the completion of requirements specified by business partners. It can happen that an application has almost no bugs but is not meeting the requirements or maybe the testing done was for the wrong requirements.

6. Early Testing

It is believed that testing would be more effective if it is initiated from an early stage. Early testing would include Documents Testing to ensure that the designing phase has no defects. Finding defects in an early stage saves a lot of money, effort and time. And so it is advised to start testing early in SDLC (Software Development Life Cycle).

7. Context dependent Testing

Testing is different for each domain. Suppose, a banking domain testing will not be the same as an Insurance domain. The methodologies, techniques and type of testing used will be different in each domain. Our last principle is focused on this factor which emphasizes the importance of understanding that the testing needs to be flexible enough to change as per the project’s requirement

How do Softwares have bugs?

As we know Software development is not an easy job. There can be many factors that may result in defects. Listed below are some of the many factors that cause defects in a Software:

• Ambiguity in Requirement: This usually happens at the time of requirement gathering. If the requirements collected are not correctly understood or not documented correctly, it might lead to an issue in the application. This can also happen on the Testing Team end, where because of ambiguity they ignore any bug during testing.
• There can also be a chance that the requirement is missed and hence not developed as well as tested. 
• If the Project model is Agile in nature: In the Agile model, the requirements tend to change. The changes are discussed and implemented parallelly. The continuous changes to the code can have the risk of leading to instability in the system or causing bugs in an already working functionality
• If the requirement is complex: Complex requirements need a lot of brainstorming. The logic needs to be thought upon before implementing. If the logics are not clear it might lead to several defects
• If the timelines are restricted: Restricted timelines take away the flexibility. A person does not get sufficient time to think and develop an understanding of the functionality. Testers usually tend to perform selective testing due to lack of time leaving defects in the application
• Lack of Skilled labours: There is a high probability that an untrained person would miss defects.

Deciding how much testing is sufficient enough

There is no benchmark comparing with which we can figure out what is enough. If we have no constraint of time, resources and cost, it will take a considerable amount of time to complete the testing. Having a 100% coverage based on the designed test cases can be defined as sufficient testing, however, it is hard to say if the designed test cases have covered each and every scenario. Hence, statistically proving how much testing is enough is a bit of a challenge. Since our scope of testing will always vary based on the below 4 factors:

Read: What is Software Testing? Type of Software Testing & Why Do We Need It?

• Timeline: Our scope of testing will be highly impacted by the timelines provided for testing. As we will have to plan our testing according to the time allotted
• Resources: When we talk about resources, if we have adequate no. of skilled resources our testing would be of high quality. Items tested by such resources would have good coverage. However, if the resources are unexperienced the testing would need more supervision and would take more time
• Cost: Cost is the driving force of the above two factors. If the company has assigned a good budget to a project the timelines can be managed by assigning more resources. However, its vice versa can happen if the budget is low.

Keeping into account the above factors we can consider our testing to be sufficient enough if:

• Our test case execution is completed and we have not encountered any issue or the count of issues is within the acceptable limit of Exit criteria
• If our requirement traceability matrix is 100% 
• If we have identified the high-risk items and taken all the required steps to mitigate the risk
• If the functionality is correctly working in UAT as well
• If the user accepts the application happily

Summary

To summarize how to increase the efficiency of Software Testing, we can say that we need to have a fair idea of the Testing Principles so that we can plan our testing in a way that it gives us a  better coverage around all the possible risks, identify the high priority features and utilizing the testing methods and types to complete the testing in the given timelines. Test Planning plays a vital role in making effective progress. It is in the hands of the testing team to strategize the testing in a way that the functionality is less vulnerable to the risk of breakdowns or finding any showstopper issue in production. Now when we say that the planning should be full-proof it is important for the testers to have a good understanding of the testing principles. As the risk can be high if the testing is poorly planned and executed. It should be disciplined enough to incorporate all the 7 testing principles as a baseline. Take into account all the factors that might lead to defects. And lastly, understand how adequate testing is!