7 Ways to Measure Test Coverage

by Nataliia Vasylyna | March 31, 2011 10:00 am

This article was updated in December 2020.

There is no one-size-fits-all guide to creating test coverage. This is why many testers could benefit from the advice on how to do it in different situations that can happen at work. They should be able to manage the time and resources in the most efficient way and concentrate testing efforts on the most relevant areas. And here is the QATestLab article to help.

Why do we need to control test coverage?

Test coverage helps us to assess the quality of testing by checking if we have tested everything that needs to be tested. A similar term is “code coverage,” a process that helps us to see which parts of code are covered by tests, and whether the minimum code coverage requirements for deploying are met.

Test coverage has many advantages. It allows us to:

• effectively set priorities in testing tasks;
• make sure that all requirements are covered;
• keep time, resources, cost and scope under control;
• prevent bugs and avoid weaknesses in the project at early stages;
• check which tests are effective and create additional tests if needed.

During test coverage, we need to make sure that we are following instructions from the QA team[1] in order to use the test cases. Another very important thing is to check whether the test cases give the necessary information to fix the detected bugs. Finally, it is useful to set up automation and special unit tools for better work.

Sufficient test coverage should include tests for various aspects of the system under test. It is important to validate software from all angles. So, testers should use different methods and test cases classification for diversification. When checking test coverage, we should pay attention to these parameters:

When it comes to measuring test coverage, there are many ways and approaches we can choose. Let’s take a look at the 7 main ones.

Possible ways to measure test coverage

1. Test coverage by feature: Check whether your tests cover all the functionality specified in the original documentation[2].
2. By a GUI icon: Each interface has its own menu, screens, buttons and pull-downs. The point of this measurement is to see if we have tests for every item of the interface.
3. By instrumentation: In code coverage, we use tools and programs that measure what percentage of code we are testing.
4. By structure: During code coverage, we test the code based on its structure and use many different techniques to do so, such as statement coverage, branch coverage, path coverage, line coverage and others.
5. By scenario: Testers check reports of user experience to see which functions don’t work properly in certain situations/scenarios. They also need to predict which extraordinary scenarios are likely to occur, and then QA experts need to create appropriate tests to check these scenarios. One of the most common extraordinary scenarios is when too many people are using the system.
6. By transition: Transition coverage is another important method used in code coverage. With this technique, we can check if the needed transition happened in our code (for example, write => read in memory).
7. By web script, page, app and component: Indication of the risk level of all these “webs” helps to lower the possibility of serious problems in the future. For test coverage to be successful, you need to see the risks of bugs.

Test coverage examples

Let’s take a look at a couple of examples to see how test coverage and different ways to measure it work when used in simple and real-life situations.

Example: “Testing a pencil”

During job interviews, candidates for a QA tester position are often asked to treat a simple pencil as a test object. If you need to test a pencil, the first thing you should check is test coverage by feature. The main functionality requirement for a pencil is to write. Then, specifications in the original documentation should tell you the color of this pencil and how soft or hard its lead is supposed to be. You need to check all of it in your test coverage by feature.

Then you can perform test coverage of your pencil by a GUI icon. In other words, you can check the design, size and surface of the pencil and decide if they are convenient for users.

You can also check this pencil by scenario. For example, you can check how hard you need to push while writing with the pencil to break it.

Example: “Testing a video player”

Similarly, you will first check test coverage for a video player by feature. All the essential features specified for this device in the requirements must work correctly.

You can also check its test coverage by a GUI icon. Is the interface of this video player convenient? Do all the elements of this interface work correctly?

Another important test would be by scenario. What happens when users do something unexpected with this video player? What if they try to use several functions of this device at the same time?

Finally, most modern video players will have some integration with web applications, so you can check this device by web script, page, app and component. What additional risks appear because of the web parts?

Conclusion

Test coverage shows the QA team if the tests checked all the necessary functions and features of the product. Test coverage is very important because we need to cover all the possibilities and avoid the risk of a fatal bug in the released system. In reality, code and test coverage rarely reach 100%. But you can make a good engineering decision on what sort of test coverage you need based on the largest and most important risks the system is prone to.

Related Posts:

• How to Provide the Best Test Coverage?[3]
• What is Test Coverage Specificity?[4]
• How to save money for buying a new car by measuring code coverage[5]

Source URL: https://blog.qatestlab.com/2011/03/31/ways-to-measure-test-coverage/

---