Code Coverage Tools: Open-Source & Commercial Compared

Code coverage measures the extent to which your source code has been tested. It is crucial for evaluating the quality of your tests and their effectiveness on your software source code, ultimately determining the quality of your project. Code coverage analysis is performed with the help of software, which is called a code coverage tool.

Over time, different tools for code coverage analysis have been developed to meet various needs, such as analysing code on a local machine (host), on the actual device (target), or both, during software development and testing. These tools are essential in software development and are available in both free and commercial versions, and each type comes with its own strengths and weaknesses.

Code coverage tools integrate easily with popular build tools like Make, CMake, Gradle, Ant and Maven. They also work well with code-quality tools like SonarQube and continuous-integration tools like Jenkins, among others. There are many code coverage tools on the market, and organisations choose them based on their specific needs, supporting different metrics, programming languages and platforms. This guide compares the most widely used open-source and commercial tools for code coverage so you can pick the right one.

How do code coverage tools work?

A code coverage tool adds measurement points or counters at build time and saves the data from those counters during the execution of tests in a runtime environment. The process of adding these measuring points is called instrumentation. In general, code coverage analysis tools work on two principles:

• Source code instrumentation. This is a very reliable method of instrumentation, because each and every line of code and condition is given a measurement point after the compiler preprocesses the source code. When tests are run against the instrumented code, execution data is collected and saved, then mapped back to the source code to generate coverage reports. As a result, you can see the code coverage of every part of your software source code, including condition and MC/DC coverage.
• Bytecode or optimized-code instrumentation. In this case, instrumentation is done on bytecode or compiler-optimized code. As soon as execution data from the runtime environment is collected, it is mapped only to the optimized parts of the source code, and coverage reports are generated from there.

Commercial code coverage tools overview

Commercial tools generally justify their cost with deeper metrics (especially MC/DC), embedded and cross-compiler support, report merging and unified reporting across large or mixed-language code bases. The table below is a quick orientation; detailed write-ups follow further down.

Tip: on a phone, scroll the table sideways to see all columns.

Table: Commercial code coverage tools overview. Prices are indicative and may change, always confirm with the vendor.

Open-source code coverage tools overview

Free and open-source tools cover most everyday host-based testing extremely well. The trade-off is that nearly all of them are single-language, run only on the host, and stop at statement and branch coverage, none of the mainstream open-source tools produce true MC/DC.

Tip: on a phone, scroll the table sideways to see all columns.

Table: Open-source code coverage tools overview.

Open-source code coverage tools in detail

Below, grouped by language, is what each free tool actually measures, where it shines and where it stops.

Code coverage for C and C++, GCov / LCov

GCov is a free, open-source code coverage tool that ships as a standard utility with the GNU Compiler Collection for the C and C++ languages. It is a statement-by-statement profiling tool. Because GCov instruments compiler-optimized code, you only see coverage for the optimized code that was instrumented. LCov is a free, open-source front end that turns GCov data into browsable HTML.

Key features:

• Generates a test-coverage analysis for optimized code.
• Tells you how many times a line was executed.
• Provides branch coverage.

Pros: Free and open-source, works with a few GCov commands, and shows the coverage rate of pure C/C++ code accurately. Cons: limited to C/C++ compiled with the GCC toolchain; shows coverage only for compiler-optimized code; flags a line as covered even when only part of it ran (no partial-line view); no real statement coverage when multiple statements share a line; no multiple-condition or MC/DC coverage for safety-critical software; no support for getting coverage from embedded target devices; cannot merge reports across runs when source has changed; and no delta or mutation coverage. Cost: Free.

Code coverage for Java and Kotlin, JaCoCo, Cobertura, JCov, CodeCover & Emma

JaCoCo works on the principle of bytecode instrumentation; you add the JaCoCo plugin to pom.xml in Maven (or to Gradle). It instruments Java bytecode on-the-fly (while running) or offline (before execution), supports all Java versions, integrates with Maven, Gradle, Ant, Jenkins, TeamCity, Eclipse, VS Code and NetBeans, and reports line, branch and cyclomatic-complexity metrics in HTML, XML and CSV. It is free and performs well on large-scale Java apps, but it covers Java and Kotlin only, requires a JavaAgent JVM parameter and manual configuration, provides no condition or multiple-condition coverage, only works for unit tests/test suites, and cannot merge reports across runs.

Cobertura is a simple, widely used Java tool based on jcoverage, runnable through Maven and Ant. It has offline instrumentation, line/branch/global metrics (plus pre-test coverage through Sonar) and HTML/XML/Text reports with filtering. It is free and easy to read, but it is Java-only, has no CSV/JSON reports, supports fewer test frameworks, and offers no multiple-condition, delta or mutation coverage.

JCov is a free tool developed with the SUN OpenJDK for Java. It instruments bytecode on-the-fly or offline and covers block, line, branch, method, field, API and pre-test metrics, storing data on the local file system or a server. It is Java-only and limited to unit testing / test suites.

CodeCover is a free glass-box testing tool under the Eclipse Public License for Java and COBOL, integrating with Jenkins, Ant, JUnit and Eclipse. It covers statements, methods, branches, loops and MCs, plus pre-test, term and synchronized coverage, with reports in Velocity format. It is excellent for improving test quality but is limited to Java/COBOL, a single source directory and unit testing.

Emma is one of the oldest, most popular Java coverage tools (under the Common Public License), though it is no longer actively developed. It instruments classes on-the-fly or offline, supports large Java apps, covers class/method/line/basic-block, reports in HTML/XML/plain text (with source linking) and can merge coverage data with no external dependencies. The downside: Java-only and lacking support for the latest Java versions.

Code coverage for Python, PyTestCov & Coverage.py

PyTestCov is a free code coverage tool for Python. Its plugin supports centralized and distributed testing (load and each mode) and subprocess coverage, exposes everything the coverage package offers, lets you skip test methods, runs tests in parallel to cut execution time, and detects test files and functions automatically. It is Python-only, needs the pytest-cov package for distributed/subprocess measurement, lacks interline detail for partially executed conditions and multiple-condition coverage, cannot merge reports when source changes, and shows no delta or mutation coverage.

Coverage.py monitors every part of your program and identifies which source was executed. You can specify the source files to analyse via config; by default it measures line coverage and generates Text, HTML, XML, LCOV and JSON reports, with an API exposing results in an SQLite database. It is free and usable via command line or API, but it is Python-only, cannot merge reports across runs when source changes, and shows no delta or mutation coverage.

Code coverage for .NET and C#, OpenCover & AltCover

OpenCover is a free code coverage tool for .NET software (.NET 2 and above; no Mono support). You install it via NuGet, MSI or ZIP; it supports 64-bit and 32-bit for .NET 4 and .NET 2, runs from the command line and reports in XML. It offers friendly documentation, but is Windows-only and .NET-only, works for unit testing (not functional testing), shows no partial coverage, and cannot merge reports across runs (no delta/mutation coverage).

AltCover is a free code coverage tool for .NET and Mono. It weaves IL for coverage, runs from the command line and generates XML, LCov and Cobertura reports on any platform with read/write access, usable via global tools, dotnet test, PowerShell and MSBuild tasks. It is limited to .NET/Mono and unit-testing coverage, with no functional testing, no multiple-condition coverage and no partial-line coverage.

Code coverage for JavaScript, Istanbul & Karma

Istanbul is one of the most popular free JavaScript coverage tools (BSD license). It generates reports for unit, server-side and browser tests; covers statement, branch and function metrics by instrumenting code with line counters; can act as middleware while serving JS files for browser testing; supports Node.js unit tests; and outputs HTML, LCOV, Cobertura and more, via a command line or library. It is easy to install with terminal output, but is JavaScript-only and shows no partial coverage.

Karma is an open-source tool that provides code coverage for JavaScript. It needs some manual configuration, lets you drive the whole workflow from the command line or IDE, integrates with Jenkins, Travis and Semaphore, and reports in HTML, LCOV, Text and Cobertura. Real devices and browsers can be used and it supports many testing frameworks, but it is JavaScript-only and shows no partial-coverage report.

Commercial code coverage tools in detail

Where open-source tools stop, commercial tools pick up: deeper metrics, embedded and cross-compiler support, report merging, and reporting that scales across large or mixed-language code bases.

Bullseye

Bullseye is a commercial code coverage tool by Bullseye Testing Technology for C and C++, used to improve software quality in the medical, aerospace and defense sectors, with limited compiler and cross-compiler support. It works for C/C++ applications including system and kernel modes in supported environments, reports in HTML and XML (importable into the Bullseye GUI), and adds code excluding, merging and visualizing. It offers high-speed execution, low renewal fees ($200/year) and a simple workflow. However, it is limited to C/C++, does not support several compilers and cross-compilers (e.g. Freescale, Keil uVision, Renesas, LLVM, Intel C++, Microchip), only merges results at the function level (not statement or condition level), cannot merge reports when source changes, and shows no delta or mutation coverage. Cost: $800 per license for the first year, $200 renewal.

Squish COCO

Squish Coco is a code coverage tool by Froglogic for C, C++, C#, SystemC, Tcl and QML. It complements Froglogic's UI-testing tool Squish, generates HTML, Text, XML, JUnit and Cobertura reports, integrates with many build, test and CI frameworks, and covers statements, branches, conditions and MC/DC. It offers a free trial and demo, but cannot be used for Java or Python coverage, has limited compiler support, cannot merge reports when source changes, and shows no delta or mutation coverage. Cost: Price not available.

VectorCast

VectorCast, by Vector Software Inc., covers C and C++ and supports a wide range of compilers, simulators and processor architectures. It performs quality trends and change-impact analysis, allows parallel testing, supports statement, branch and multiple condition/decision coverage, supports unit, integration, functional and embedded target testing, automates regression testing, provides a web-based quality dashboard, and creates unit and integration tests for C/C++ (suiting Agile and TDD). It offers a free trial and automatic test creation from decision paths, but cannot be used for Java or Python coverage, has limited compiler/cross-compiler support, does not support resource-limited embedded systems, cannot merge reports when source changes, and shows no delta or mutation coverage. Cost: Price not available.

Parasoft

Parasoft C/C++test, Jtest and dotTest provide code coverage for C, C++, Java and .NET. The solution covers lines, statements, blocks, decision/branch, paths, MC/DC and conditions, generates unit and integration coverage mapped to source in the Eclipse IDE (so you can edit source directly), and runs from the IDE or command line. You can use it with all build tools, CI tools and IDEs, and its pictorial coverage view is user-friendly. On the downside, it is geared to large, traditional organisations, is expensive (not free), cannot merge reports when source changes, and shows no delta or mutation coverage. Cost: Price not available.

Cantata

Cantata, by QA Systems, is an automated testing suite for C and C++. It supports almost all coverage levels, offers unparalleled coverage filtering, generates HTML and XML reports, lets you integrate coverage analysis with Cantata tests or any other tests, and generates unit and integration tests. It cuts the cost of standards compliance, lowers testing costs and reports failures clearly for complex applications. However, it is limited to C/C++, does not support all compilers and cross-compilers, cannot merge reports when source changes, and shows no delta or mutation coverage. Cost: Price not available.

NCover

NCover, by Peter Waldschmidt, is a code coverage tool for .NET applications. It reports statement and branch coverage, provides centralized coverage reports, ships extensive documentation and user support, works well in agile environments and presents reports in HTML. It is available under a commercial license as well as open source, and tests can be run manually or automatically. Its main limitation is that it covers the .NET platform only. Cost: $480 to $658 per year.

DotCover

DotCover, by JetBrains, is a commercial code coverage tool for .NET applications. It integrates with Visual Studio, performs statement-level coverage for .NET, Silverlight and .NET Core, provides a console utility for a continuous-integration server, and lets you customize analysis with a coverage filter. Its main limitation is that it covers the .NET platform only. Cost: $11.90 to $39.90 per month.

How to choose a code coverage tool

Consider the following factors when selecting a tool for code coverage analysis. Regardless of how long a vendor has been in business or how many clients they have, you should always perform a proof of concept (POC) before committing.

1. Language support. Is the tool compatible with all the programming languages used in your organisation? A single cross-language tool pays off when you span C, C++, CUDA, JVM, Python, C# and JavaScript.
2. Code coverage for C or C++. If you need code coverage for C or C++, does the tool support all compilers, cross-compilers and embedded targets? Check whether it will keep up with future compiler or cross-compiler updates used during development.
3. Ease of use. The tool should integrate easily with your existing development and testing environment, so a developer or tester can generate coverage with just a few extra steps.
4. Embedded readiness. When developing for embedded systems, the tool should auto-detect and add the right run-time libraries to your build based on the embedded target architecture. See code coverage for Keil and code coverage for IAR for examples.
5. Metric depth. Decide whether line/branch is enough or whether you need condition, multiple-condition or MC/DC coverage, the latter is mandatory for safety-critical work under DO-178C and ISO 26262.
6. Support quality. No matter what a company claims about its support, judge for yourself how quickly they resolve issues during the POC.

Frequently asked questions

Conclusion

Many code coverage tools are available today. The important thing is to pick the right one for your project, one that meets all your requirements without limitations and comes with good technical support, rather than a tool that only works in selected environments. For everyday host-based, single-language testing the open-source options are genuinely excellent and free; the moment you span multiple languages, ship to embedded or GPU targets, or need MC/DC for safety-critical work, the calculus shifts toward a tool built for that breadth and depth.

So don't forget to check out the RKTracer tool to ensure quality code coverage, and ultimately quality software projects. Start a free trial or book a demo to see it run on your own build.