A guide to code coverage, part 1: Code coverage explained
It’s not easy being a software engineer — even though you’re trying to do the best you can, chances are you mess up from time to time. Of course that’s not the end of the world — you’re only human, and humans make mistakes. But wouldn’t it be great if you would have to worry a little less about cleaning up after yourself?
Codacy is the perfect code review partner for developers, and that’s why we’re publishing a two-part series about one of the most important concepts in programming quality assurance — code coverage.
In this first post, we’re going to take a closer look at the concept of code coverage itself — what it is, why is it important, and how can it be used to optimize your code.
The basics of code coverage
As a programmer, you can’t just write code and and hope every line is flawless — you’re bound to make mistakes, and that’s why testing is such a critical part of the development workflow.
In the most basic sense, code coverage is a way of using analytics to get an idea of how well an application has been tested. Your tests might return positive results across the board, but if they only cover 30 percent of your code it’s hard to be confident about the end product.
That’s where code coverage comes in — the higher the value, the better, as a more thoroughly tested application generally has less show-stopping bugs or other flaws.
Calculating code coverage
There are a couple of ways to determine how well your code is covered by the tests you run. These various metrics all offer a different perspective on your code quality, and it’s useful to know the basics of each of them — that’s why we’ll run them down for you.
At the heart of code coverage there’s statement coverage, which checks how many statements in your program have been executed. This is the most widely used form of code coverage as it is found in most of the relevant code coverage tools.
Another fairly basic example of code coverage. Especially important in applications that rely on a large number of functions, this coverage metric focuses on how many of the declared functions have been called during testing.
As a higher-level way of checking your code, the main reason to use branch coverage is to see how many branches of every control structure have been properly executed. If a program is able to jump, it should jump to all possible destinations — if there’s an ‘if’ statement, it checks if both the ‘true’ and ‘false’ branches have been executed.
If your tests achieve full branch coverage, your app is protected against errors in all branches, which means that 100 percent branch coverage also indicates 100 percent statement coverage.
This metric looks at the various boolean sub-expressions in your code and if they were tested for both ‘true’ and ‘false’. Because expressions can get complicated, it’s hard to achieve 100 percent condition coverage — that’s why there are multiple ways of reporting this metric, all of which focus on covering the most important combinations.
Other kinds of coverage
While these are the most important metrics, there are others to keep in mind when thinking about code coverage for your testing. One of these is decision coverage, which is a combination of function coverage and branch coverage that checks if all entry and exit points, and all decisions and outcomes have been invoked at least once. Another metric is multiple condition coverage, which requires all combinations of conditions inside decisions to be tested, effectively looking for full decision and condition coverage.
Then there’s a host of less-important coverage metrics like linear code sequence and jump coverage, path coverage, entry/exit coverage, loop coverage and stage coverage. Because this article is meant to only offer a look at the basics of code coverage, we won’t dive into these — but it’s still good to know they exist.
How to apply code coverage
All in all, code coverage is an important way to see if your tests are comprehensively covering your code. In a large company the above metrics would be used by a team of Quality Assurance employees, but for startups and solo developers you should be looking out for them yourself.
It’s vital, however, that you stay critical of your code even when you achieve 100 percent coverage on the various metrics — while it can be a great way to indirectly check its quality, it relies on knowing what to test in the first place.
In the next article of this two-part series we’ll take a look at how to set up code coverage in Codacy, and how it offers a smart way to analyze your code.
Don’t worry, get Codacy
At Codacy, we’re on a mission to make your life as a developer much easier by building the world’s most powerful platform for code review. With an easy-to-use interface and comprehensive analytics, you’ll be shipping top-notch code in no time. Get started with our free trial today!