Sanity Testing

In this tutorial, we are going to study Sanity Testing, which is a type of software testing usually performed after the completion of smoke testing to ensure the correctness of the software build. Earlier, we studied other types of software testing such as Manual, Automation, Regression, and Smoke testing.

Here, we will look into Sanity Testing, the need for Sanity Testing, characteristics of Sanity Testing, attributes of Sanity Testing, comparison between sanity and smoke testing, advantages and disadvantages of Sanity Testing, and more.

What is Sanity Testing?

A type of software testing performed on the stables builds of the software is known as Sanity Testing. It is usually done after the successful completion of smoke testing. Sanity testing is a subset of Regression testing performed on the significant or critical functionalities and features of the system. It provides a narrow and deep coverage of testing specific software modules. It usually ensures that small software functionality is working as expected after some minor changes.

The goal of sanity testing is to test the small functionalities of the system are working accurately against the requirements even after code modifications and defect fixes are done to the system build. The software build should not be impacted by these modifications, and no other bugs are encountered in the software build.

Sanity testing is a kind of Functional testing which tests the proposed functionality of the software against its functional requirements. In other words, a set of preliminary and quick tests is performed to validate the behaviour and working of the software’s components. Sanity testing is similar to Smoke testing, although both testing types differ in many aspects. We will compare sanity and smoke testing later in this section.

Need for Sanity Testing

In software testing, sanity tests are performed on the software builds to validate and verify the essential features and modules of the application under test. After the completion of smoke testing, the build is accepted by the testers for performing other testing activities. Sanity tests are usually performed in scenarios such as when new functionalities are added, code modifications are done, or bugs are fixed. Such changes made in the smaller sections or specific modules of the software are tested under sanity testing as it is a sub-division of regression testing which works for particular features having modifications.

Some of the critical points on the need for sanity testing are mentioned below:

• One of the critical requirements of sanity testing is that the planned modules and features should work as expected.
• Sanity tests should be performed when the software build is moved to the QA environment after a bug or issue has been resolved by the developers.
• To test the rationality and accuracy of the software and check whether it is ready for further testing.
• To test the newly added features or modules and modifications done in the existing build of the software.
• To validate the core functionality of the software, but not as exhaustive as regression testing.
• It also helps in identifying the stoppage of the software build.

Examples of Sanity Testing

Following are the cases in which we need to perform sanity testing:

Example 1: Suppose there is a mobile banking application, which consists of various modules such as a login page, dashboard, statement page, debit card information, loan information, etc. However, in the first version or build, we focus on a few basic features like login of an existing user, dashboard or home page, and statement page.

The requirement specified by the clients for the login page is that only existing bank customers or users should be able to log in. Random users and users who don't have an account with the bank should not be able to use the application. If a new user tries to log in on the application, a popup should appear with an error message stating “User not Found”.

While testing the application, testers found a bug in the login page, i.e., when they try to log in with a new username/ mobile no. and password, they can log in successfully, and no error appears. This issue is reported to the development team for a fix with the proper evidence. Once the development team fixes the issue, it is returned to the testing team for testing. Then, testers test the same functionality to ensure that the issue is being resolved and that particular feature is working fine. Testers also verify the other functionalities in the software version or build are not impacted by the modifications and working as per the requirements. Hence, as part of sanity testing, the tester has validated that the code changes and modifications are done correctly, and along with that, other functionalities are remained untouched.

Example 2: This example is similar to the above one. On an e-commerce website, we are testing the features of adding the items into the cart and proceeding with checkout. Here, testers identified a bug while proceeding with the checkout. While giving the mobile number, the text field accepts numbers of more than ten digits which is a significant bug. In the requirement for the mobile text field, it is mentioned that the text field will only accept ten numeric characters and alphanumeric, alphabets, and special characters are not acceptable. Once the bug is confirmed, it will be reported to the developers for a fix with a detailed report and evidence. When the bug is resolved by the developers, it will be reported back to the testers for retesting. The testers again perform sanity tests on the same feature and other functionalities present in the existing build to verify that the issue has been fixed and other functionalities are not impacted by the code changes and defect fixes. Hence, as part of sanity testing, the tester has validated the functionalities of the software build, identified bugs and issues, and again tested the code changes and modifications to ensure that these changes are not affecting the other functionalities of the software.

Characteristics of Sanity Testing

Sanity testing is a process of testing specific critical functionalities of every software build in the early stages of the testing cycle. It verifies that the deployed software build is reasonable and can be moved forward to further testing. Some of the characteristics of sanity testing are mentioned below:

• Sanity testing is performed on the software build after smoke testing is done to test the code modification, newly added functionalities, and bug fixes.
• As it is a subset of regression testing, we can test the particular functionalities and features of the application roughly.
• Sanity testing is a quick and time-saving process compared to regression testing because it doesn’t allow us to perform detailed testing.
• It provides deep and narrow coverage of the particular module which is under tested.
• The main characteristic of sanity testing is that it helps us understand the capability of the developed software built for performing rigorous testing.
• When a bug is identified while executing sanity tests, testers will reject the software build due to bugs, and it will help in saving time and human efforts for rigorous testing.
• Sanity tests do not have any future reference as it is scripted and can’t be documented.

Attributes of Sanity Testing

For a better understanding of sanity testing, there are some features and attributes available in software testing which enhance the sanity testing process. Below mentioned are the elements and attributes of sanity testing:

• Simple and Quick – Sanity testing is simple and quick in comparison to other testing types because they are efficiently designed and created and takes less time in test execution. The process of feedback is also fast in sanity testing.
• The subset of Regression testing – As discussed, sanity testing is a subdivision of regression testing which is not as intense as regression testing and focuses on a particular feature or functionality of the software.
• Not Documented – Sanity tests are never documented. Therefore, we don’t have any references for sanity tests in future.
• Deep and Narrow test coverage – Sanity testing provides a narrow and deep testing coverage where limited modules are deeply protected.
• Unscripted – We all know that sanity testing is not documented. Similarly, sanity tests are primarily unscripted.
• Performed by Testers – Sanity tests are usually performed by QA engineers or Testers.
• Comprehensive testing – Sanity testing is said to be comprehensive enough as it covers smaller sections of the software, but they give full coverage of that particular section.

Sanity Testing Process

Sanity testing is a basic, simple, and quick testing process which do not have any complex procedures and techniques. To validate the issues and incorrect outcomes of the software build, we have to follow some basic steps to proceed with sanity testing.

Following are the steps and procedures we implement to make the sanity testing efficient:

• Identification of the changes made to the system – This is the first step of the sanity testing process. In this step, testers will examine the software and its requirements to identify what exactly needs to be tested. The identification involves newly added features and functionalities and modifications done in the code while resolving the bugs. The purpose of this step is to identify all the testable items of the software build.
• Evaluation – After completing the identification step, we move forward to the second step, which is evaluating the testable items. The result of step 1 will be the input for step 2, i.e., identified features and functionalities, system modifications, and fixed defects. Testers will evaluate all the above items to test their intended and proper functioning.
• Testing – The third and last step of the sanity testing process is testing which is performed after the completion of the evaluation step. Testers will start testing all the evaluated items against the requirements specified in the document. Testing will cover all possible parameters and bases related to the testable items to ensure the behaviour and working of the software. If the testing is successful, the software build will be passed and accepted for further detailed and rigorous testing.

Difference between Sanity Testing and Smoke Testing

Following are the differences between Sanity and Smoke testing:

Sanity Testing	Smoke Testing
Sanity testing is performed to test the small functionalities of the software after any code modifications or defect fixes.	Smoke testing is performed to test the stability and correctness of the latest software build.
Sanity testing is done on the stable builds of the software.	Smoke testing is done on unstable or latest software builds.
It is a sub-division of Regression Testing.	It is a sub-division of Acceptance or Build Verification Testing.
It provides narrow and deep test coverage for testing the application, i.e., specific modules of the applicated are tested.	It provides broad test coverage for testing the application, i.e., the overall system can be tested.
It is usually done to measure the rationality of the system.	It is usually done to measure the stability of the system.
Sanity tests are performed by the testers only.	Smoke tests can be performed by testers or developers but are usually done by the testers.
Sanity tests cannot be documented.	Smoke tests can be documented in comparison to sanity tests
Testing covers particular modules which have been modified.	Testing covers the end-to-end functionality of the software.

Difference between Sanity Testing and Regression Testing

Following are the differences between Sanity and Regression testing:

Sanity Testing	Regression Testing
Sanity testing is performed to test the specific functionalities/ features of the software build after some modifications are done to the build.	Regression testing is done to test the entire system after any code changes or defect fixes are done to the system. It also verifies the system should not be impacted by such modifications.
Sanity testing is not planned testing and is carried out within time constraints.	Regression testing is planned testing.
It is a subset of regression testing.	On the other hand, regression testing is an independent testing type.
It provides narrow and deep test coverage for testing the software, i.e., specific modules of the applicated are tested.	It provides broad and deep test coverage for testing the software, i.e., all modules or the system being tested.
Sanity tests are usually performed by the testers.	Regression tests are also performed by the testers via automation.
Sanity testing usually does not use scripts for testing.	Regression tests usually use scripts for testing.
Sanity test cases are not always prepared or created, and a set of test cases are selected from the regression test suite.	Regression test cases are always prepared and designed for test execution.
Sanity testing is done after completing smoke testing and before regression testing.	Regression testing is done in the later stages of the testing process.

Advantages of Sanity Testing

Following are the advantages of Sanity testing:

• Sanity testing is a basic testing process which is very easy to understand and implement.
• It consumes less time and human effort as sanity tests are more focused on specific functionalities and modules of the software.
• It helps in the quick detection of bugs and defects in the critical features and functionalities of the software’s build in the early stages.
• It also helps us to identify issues related to deployment and compilation.
• In sanity testing, there is no need to document test results as it is unscripted.
• If a defect is identified during sanity testing, the software build will be rejected by the testers, which saves a lot of time and human effort for further testing processes such as functional, non-functional, regression, etc.
• Sanity testing is less expensive in comparison to other testing types.

Disadvantages of Sanity Testing

Following are the disadvantages of Sanity testing:

• One of the significant drawbacks of sanity testing is that it only focuses on the commands and functions of the developed software.
• It covers some particular functions and features of the application and doesn't allow us to test other software functionalities. Therefore, issues and bugs of other functionalities remain unchecked.
• As we don’t document sanity tests, we do not have any references for sanity testing in future.
• Sanity testing doesn't cover a structure-level design, and the identified bugs will be difficult to understand and fix for the developers.
• In sanity testing, it isn't easy to cover all possible test cases for a particular functionality, as a set of test cases is used to perform testing.