Bugs or Defects in Software Testing

In this tutorial, we will study software testing defects or bugs in detail. We will walk through the terminologies related to defects and how they occur in the system, the types of defects, and defect management tools.

As we have discussed earlier, testing is performed to ensure the quality standard of the product. It also helps in delivering a defect-free product that meets the requirements of clients and customers. While testing, we may encounter some bugs that deviate the software’s functionality and performance from the expected one.

What is a Bug or Defect?

In software testing, a bug is introduced as a difference between the client’s requirement and the developed software. In other words, a defect is an error or flaw in the system that deviates the software from its expected results. If a system has bugs and errors, it will produce incorrect or unexpected outputs that can cause system failure and crashes. To avoid system failure and crashes, testing is performed to identify all possible bugs in the system.

If a tester is not getting the desired result while executing the test cases, there might be some issues in the application under test, which are called Defects. Once the defects are found, they are moved or assigned to developers for a fix. There is a complete life cycle of a defect, starting from when it's found to the end the defect. Other terms for a bug are defect, error, issue, or problem in software testing.

Defects usually occur in the system when developers make mistakes while developing the software. It can be due to some missed code, code overlapping, etc.

In software testing, we have several types of errors or mistakes which are generated while developing the software, as mentioned below: -

• Syntax Errors – This type of error occurs when a programmer makes some mistakes in the writing style of code. A minor fault can also lead to a syntax error in the code, such as missing a semicolon or curly braces at the end of a line. Syntax errors can be corrected while running the program as an error will be thrown instead of the result.
• Arithmetic Errors – It occurs when a developer makes mistakes in arithmetic expressions or conditions while developing the code of the application. Code congestion may also lead to arithmetic errors as it is hard to read or watch a lengthy program. Once the code is deployed, errors will impact the developed application, and while testing, the actual output will differ from the expected.
• Logical Errors – This type of error occurs when a developer makes mistakes in the logical implementation of the code. Sometimes, due to complex requirements, developers might get confused or not understand the requirements clearly. Logical errors are usually associated with the core of the application.
• Performance Errors – It occurs when a system is inadequate to meet the expected results. Sometimes, a system may not be efficient to fulfil all the requirements (non-functional). Then defects related to performance, such as load, stress, reliability, usability, etc., occur in the system.
• Functional Errors – This type of error occurs when a system is inefficient in meeting the desired outputs per the functional requirements specified by the client. E.g., On a login page, if we click on the submit button and it won't navigate to the expected page, it is known as a functional defect.
• Interface Errors – This type of error occurs when an end-user interacts with the user interface of the application. The reason behind the occurrence of errors might be the interface is not user-friendly, the look and feel of the interface, or the platform dependency of the interface.

Defect Terminologies

In software testing, we have various terms related to defects or bugs. Some of the basic and important terminologies are mentioned below:

Defect – A defect is an error or flaw in the system that diverges the product or application from performing its intended functions and results in incorrect or unpredicted outputs. Due to the defect, the application will not work as per the requirements.

Bug – A bug is acknowledged as the informal name of a defect that makes the system operate incorrectly.

Fault – A defect is also acknowledged as a fault.

Error – An error is referred to as a programming or coding mistake made by the developers while developing the software. Errors are usually generated in the source or internal code of the application. It can be caused by misunderstanding the requirements or misconceptions about the working module.

Mistake – An error is acknowledged as a mistake. A mistake is generally found in the source or internal code of the application. It is a human error.

Issue – The term issue is a mix-up of all terms such as bugs, errors, mistakes, etc. Hence, it can be anything from a change request, incorrect or missing requirements, or newly added tasks.

Failure – When there are multiple bugs in an application, the system fails to perform its expected functionality which leads to a system failure or crash. Sometimes, a system is not compatible with the other devices, which results in system failure.

Bug Severity – Bug severity is defined as the impact of a bug on the application. In other words, it is the degree of impact a defect creates on the quality of a system or a product. Basically, it is a classification of defect or bug to indicate the impact degree created by a defect on the operation and working of the system. It is one of the most important defect attributes, which are classified into five categories, as mentioned below: -

• Blocker – It is a type of bug that blocks the entire system due to its impact. When a system encounters a bug with a blocker severity, the testing process is stopped by the blocker, and testers cannot proceed with the testing of other modules. The reason behind the blockage could be the system crashes in a particular environment (either testing or developing).
  The blocker is further classified into two types: -
  1. Blocking a crucial flow – When a major flow is not working, e.g., a registration followed by a website login is not working as expected.
  2. Blocking a crucial feature – When a major feature is not working, e.g., we can add items in the cart, but the cart itself is not opening, getting some error while opening the cart.
• Critical – It is a type of bug that affects the critical functionality or critical data of the application. When a critical bug is encountered by the system, it means the critical functionality of the system is not working properly. The "Critical" bug is different from the “Blocker” bug as the blocker stops the testing completely, whereas critical impacts the critical functionality, but it doesn't block the testing.
• Major – It is a bug that affects the major functionality or data of the application. When a major bug is encountered by the system, it means some of the major functionalities or supporting components/ modules are not working, but it won't stop the testing. Meanwhile, the testing team can continue testing other modules.
• Minor – It is a type of bug that affects the minor, basic or non-critical functionality of the application. It doesn't influence the working of the system, and testing can also be performed with minor severity bugs.
• Low or Trivial – It is a bug that doesn't impact the system's functionality, productivity, and efficiency. Usually, it is related to the UI and its look and feel. Trivial bugs won't cause a system failure or breakdown.

Defect severity can also be denoted as

S1 – Blocker, S2 – Critical, S3 – Major, S4 – Minor, S5 – Low

Bug Priority – It is defined as the order of fixing defects by the developer. Bug priority is an important attribute of a bug as it determines the order in which a bug gets fixed and how fast it should be fixed. If the defect priority is low, it will be fixed later or at last. Similarly, if the priority is high, the defect will be fixed as soon as possible by the developers. There are different levels of defect priority in software testing, as mentioned below: -

• Urgent or Critical – If a defect has priority as "Urgent" or "Critical", the defect must be fixed before any other defect such as high, medium, or low. This type of defect has the maximum impact on the system.
• High – If a defect has "High" priority, it should be resolved as soon as possible after fixing the critical defects. A “High” priority defect is also affecting the system severely.
• Medium – If a defect has a "Medium" priority, it can be resolved parallelly to other development activities as it doesn't affect the software much compared to high and critical defects. It can also be fixed in the next release or until the new version is created.
• Low – If a defect has a "Low" priority, it can be fixed only after all the other priority bugs are resolved or can be deferred for the next release. "Low" priority bugs don't affect the productivity of the system, so they can be remained unfixed or resolved at last.

Defect priority can also be denoted as

P1 – Critical, P2 – High, P3 – Medium, P4 – Low

Defect severity and priority are assigned by a tester in consultation with the project manager, test manager, or client. Sometimes, developers feel the assigned severity is incorrect, so they can update or modify the severity by discussing it with testers and managers. While modifying the severity, they must update the comment and the reason behind it. Only severity can be modified by developers, and priority cannot be modified once assigned to a bug.

Why do defects occur?

There are multiple reasons behind the occurrence of defects or bugs in a system. Some of them are mentioned below: -

Ambiguous or Unclear Requirements – One of the key factors for generating defects is unclear requirements. When the requirements and specifications are not completely understood by managers and developers, this may lead to the generation of multiple defects in the system. To avoid this, the team (manager, developer, and testers) should thoroughly study and examine all the requirements specified by the clients until they get a clear understanding of them. Once the requirements are understood, static testing techniques (Reviews, Walkthrough, Inspection) should be performed to ensure the understanding of the team.

Lack of Communication – It is one of the major reasons behind the occurrence of defects in the system. It happens when there is a lack of communication and coordination within the team or with the client, which leads to the misinterpretation of the system’s working and productivity. In some cases, when the designed software or product is completely different and new, the client itself is not sure about the functionality of the system and conveys the same to the development team. Therefore, both the client and development team remain confused and unclear about the product in such situations, which leads to several misinterpretations.

Programming or Coding Errors – It is defined as the coding errors and mistakes made by the developers while designing and developing the software. There are mainly three sub-sets of programming errors which are Missed code, Wrong code, and Extra code.

1. Missed code – It means that the developer, by default, missed or ignored a code snippet that is required for developing a particular module. E.g., Some items are added to the Wishlist, but when clicked, it opens a blank page, and the same happens when we try to click on the Wishlist link. Hence, due to some missing code, that particular module is not working properly.
2. Wrong code – It means that the developer has not implemented the code properly. In simple words, the wrong code has been implemented by a developer. E.g., On an E-commerce website, we are searching for tablets, and the search result shows mobile phones instead of tablets. Hence, due to wrong coding or wrong keywords, the result is coming incorrect.
3. Extra code – It means that the developer has done some extra coding which may result in some new features or functionality for a particular module. The extra code snippets are not required as per the client's requirements. E.g., there is a signup page in which only user id and password fields are required to be developed per the requirements. Hence, due to extra field or any extra code which is not required as per the client will lead to unwanted efforts.

To avoid coding errors, a complete code review should be done once the coding is completed. Unit testing also helps in reducing coding errors.

Strict Deadlines – A strict deadline can also generate bugs in the system. In some cases, when the client wants an early delivery or provides less time for developing and testing the overall product, the chances of encountering bugs get increase. Sometimes, while testing the system in the hustle, bugs are not identified in the system and may lead to failure of the system. To avoid such situations, proper planning and estimation must be done within the team.

Software Complexity – When the client’s product is very complex to design and develop, it may lead to the generation of bugs in the system. To avoid such a situation, the team should thoroughly examine all the requirements specified in the documents and case of any queries, they should clarify them with the client and product team.

Defect Tracking Tools

There are various defect tracking tools available in the market for bug reporting and tracking. Some of the most popular tools are listed below: -

• JIRA – It is one of the most popular manual testing tools, which is mainly used for project management and defect tracking. It is also recommended for agile projects as it is easy to use and operate. JIRA is an open-source bug tracking tool which can be easily downloaded from the internet. It provides various features such as workflow, reporting, recording, etc. Basically, it connects both developers and the tester to keep track of the logged defects.
• Katalon TestOps – It is an open-source, free and powerful defect tracking tool developed by Katalon studios. It is mainly used for important features such as orchestrating tests, improving workflow, connecting DevOps, etc. TestOps helps testing and DevOps teams in clearly understanding the requirements and tests, the connection between the tests, and resources used to perform testing, and the test environment. It is compatible with almost all testing frameworks (JUnit, Python, Mocha, Jasmine, etc.), CI/CD tools (Jenkins, CircleCI), and management platforms (JIRA, Slack). It also provides fast and accurate debugging of the system.
• Backlog – An online defect tracking and project management tool mainly used by the development teams of various organizations. It provides the complete history of reported bugs and issues, such as bug updates, changed status, comments, etc., and makes bug reporting very easy. Finding defects on the Backlog tool is an effortless task with its search and filters. It is also used for managing projects with additional features such as Gantt and burndown charts, IP access control, sub-tasking, Git and SVN repositories, etc.
• ClickUp – It is primarily a project management tool used by several organizations. Along with a powerful project management tool, it is also a strong defect tracking tool that helps a tester create customized views from the ground up. It consists of various features such as advanced search options, a huge variety of templates, automatically importing documents from other apps., managing and assigning tasks, quick collaboration with other teams, etc.
• BugHerd – It is the easiest tool used for tracking bugs and managing website responses. It helps the team and customers find the exact issue by searching the pinned feedback on the web pages. It also helps in quickly replicating and resolving the bugs by capturing the required information such as CSS selector data, operating system, screenshots or snapshots, and the browser. All information, including defects, responses, and technical data, is provided to the Kanban-style task board for assigning, managing, and resolving the issues. It can also work with existing or other management tools to make the defect fixing process easy.
• Marker.io – Marker is also a defect reporting and tracking tool mainly used for visual annotations. It basically tracks the bug directly on live websites with visual reporting and is primarily made for the software development teams and agencies. It helps the developer gather feedback or bug reports with all information such as operating system, page URL, browser information, console logs, screenshots, technical data, etc. After collecting the information of the bug report, it sends the data to compatible project management tools like Jira, Asana, GitHub, ClickUp, Trello, etc.
• SpiraTeam – It is primarily a quality application lifecycle management (ALM) system which is also used for defect tracking functionality. It helps the testing team handle the testing process completely, from the requirements gathering to test case development, defects, and issues with built-in traceability. Like other bug tracking tools, it is also an easy-to-use tool with various features such as reporting bugs and issues via email, automatic creation of incidents while test execution, linking bugs to other incidents and artefacts, etc.
• Zoho bug tracker – It is one of the strongest defect tracking tools that help the development and testing teams to scan or see the issues and defects based on priority and severity. Under the Zoho project, there are several modules, one of which is the Zoho bug tracking feature. It is mainly used for creating projects, tasks, milestones, defects, documents, and reports. Unlike other tools, it is a commercial tool. It also provides a feature of automated reminders and notifications for all team members.
• Userback – Like Marker, Userback is also a visual defect reporting tool. It is mainly known for its fastest bug reporting from any website or application. It is also used for quickly collecting feedback from websites and web pages. Reporting bugs with visuals and highly contextual data is an effortless task for the Userback tool as it is an easy-to-use tool. It can automatically capture recordings, videos, images, screenshots, event tracking, browser information, etc.
• Bugzilla – It is also a manual testing tool mainly used for bug tracking. The main purpose of using Bugzilla is that it keeps track for its users and clients of all the issues and defects reported on it. It is an easy-to-use tool with a good web-based interface. It maintains a link between Bugzilla and other test case management tools; therefore, it is also referred to as a project management tool. Bugzilla is also an open-source testing tool which provides a wide range of functionalities like email notifications for modified code, time tracking, strong security, advanced search options, customization, reports and charts, and localization. It is developed in Perl language and uses the MySQL database.
• MantisBT – A manual testing tool mainly used for defect reporting and tracking the application under test. The main function of Mantis is that it provides a mobile version (Android and iPhone) for testing, along with its compatibility with various operating systems such as iOS and Windows. It is one of the easiest tools available in the market as it provides a user-friendly GUI over a period. Mantis is an open-source testing tool like other tools. It is developed in PHP and operates or works with various databases such as MS SQL, MySQL, and PostgreSQL. Some of the basic features of Mantis are notification alerts, full-text search, plug-ins, audit trails for the modifications, etc.
• Micro Focus/ HP ALM or Quality Center – It is an end-to-end application lifecycle management (ALM) tool which is used by various organizations for managing projects and applications. The defect tracking system is an integrated feature of the HP ALM tool with a simple and efficient defect tracking mechanism. It is one of the most expensive commercial bug tracking tools available in the market, which doesn't support all web browsers. It is also used for multiple agile projects.
• Trac – An open-source tool primarily developed for tracking issues in the system. The focus of Trac is issue management instead of bug tracking. Trac is mainly used with other Source Code Management (SCM) tools for browsing the code, viewing modifications and history, etc., of the issue or incident. It is developed in python language for web-based applications. It is also a platform-independent tool and supports multiple platforms such as Windows, Unix, Mac OS, and Linux.
• Redmine - Like Trac, Redmine is also an open-source bug tracking tool which collaborates with other SCMs. It is mainly developed for managing web-based applications and projects, and bug tracking is a supporting module. Redmine is also a platform-independent tool which supports multiple databases and works on the Ruby framework. Some of the major features of Redmine are managing files and documents, the flexibility of issue tracking and role-based access control, supports for multi-languages and databases, using Gantt charts and calendar, etc.
• Monday – Primarily, it is a project management tool but is also used as a bug tracking tool. It helps the team examine and enhance the performance and manages the team in one place. It is mainly used for enhancing bug reports, automating various tasks and activities such as daily workflow, collaboration with Gmail, Google calendar, and MailChimp, tracking defect status, etc. It also provides an easy-to-use dashboard for data visualization.