Code Coverage

Code Coverage:-


Code coverage is a measurement of how many lines/blocks/arcs of wer code are executed while the automated tests are running.


Code coverage is collected by using a specialized tool to instrument the binaries to add tracing calls and run a full set of automated tests against the instrumented product. A good tool will give us not only the percentage of the code that is executed, but also will allow us to drill into the data and see exactly which lines of code were executed during a particular test.

Link to Use the Code Coverage template to store the average code coverage for a particular cycle time:- Code Coverage Template

Code Coverage in SonarQube :- SonarQube is an open source static code analyzer, covering 27 programming languages. It helped to standardize coding standards and write clean code, making sure no code with code smells goes to production.

Working with jacoco-maven-plugin:-

If we are using Jacoco for a code coverage report for a maven project we need to set up jacoco-maven-plugin in pom.xml file. To configure jacoco-maven-plugin, er need to add these lines in the <plugins> section of pom.xml:


<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>0.8.1</version>

</plugin>


Jacoco-maven-plugin provides many maven goals to configure for the project. Let’s have a look at frequently used maven goals for the plugin:


  • Prepare-agent: prepare-agent goal is important to setup the runtime agent for code coverage. This goal sets some important parameters and argLine which are required to run the agent and prepare execution info for the code coverage report. With this goal we can configure how to generate the execution info file for the coverage reports. Here is the minimal configuration required for this goal:


<execution>

<id>default-prepare-agent</id>

<goals>

<goal>prepare-agent</goal>

</goals>

<configuration>

<destFile>${project.build.directory}/jacoco-ut.exec</destFile>

</configuration>

</execution>

As shown above we need to give a unique id to all wer execution goals and for the prepare-agent goal, we need to configure the destination of the execution file with <destFile> configuration.

  • Prepare-agent-integration: If we have integration tests setup for a project then we need to configure how execution file gets generated for these tests, similar to shown above for unit tests. Make sure we provide a different path for execution file or set <append> configuration field to true to append execution info for integration test to existing execution info file.

  • Report: This goal is important to generate the code coverage reports in multiple formats (HTML, XML and CSV) from execution info. In this goal we can configure classes to exclude from the report or classes to include and output directory and encoding of the reports. Make sure <dataFile> configuration is pointing to execution info file generated in prepare-agent goal. Here is the minimal configuration required for JUnit test code coverage:

<execution>

<id>default-report</id>

<goals>

<goal>report</goal>

</goals>

<configuration>

<dataFile>${project.build.directory}/jacoco-ut.exec</dataFile>

<outputEncoding>UTF-8</outputEncoding>

<outputDirectory>${project.build.directory}</outputDirectory>

</configuration>

</execution>

  • Report-integration: Similar to the previous goal, we can provide configurations for generating code coverage reports in different formats for wer integration tests. If we have configured a different execution file path for integration test then make sure to use it as a <dataFile> in the <configuration>.

  • Check: This is an important goal to automate wer builds by applying constraints based on the code coverage. With the check goal, we can configure rules for wer project to meet on code coverage, for example more than 80% code coverage, otherwise wer build will fail. Here is the configuration for the same:

<execution>

<id>default-check</id>

<goals>

<goal>check</goal>

</goals>

<configuration>

<dataFile>${project.build.directory}/jacoco-ut.exec</dataFile>

<rules>

<rule>

<element>BUNDLE</element>

<limits>

<limit>

<counter>INSTRUCTION</counter>

<value>COVEREDRATIO</value>

<minimum>0.80</minimum>

</limit>

</limits>

</rule>

</rules>

</configuration>

</execution>

we can apply many more rules for wer build based on the code coverage report. we can read more about these rules on plugin’s documentation: https://www.eclemma.org/jacoco/trunk/doc/check-mojo.html#rules

Once we have configured the jacoco-maven-plugin, we need to run mvn clean install phase to generate the code coverage report. With above configuration, code coverage report must be generated in the build directory(target) in different formats (jacoco.xml, jacoco.csv and index.html).

Integrate Code Coverage with SonarQube :- Once we have the coverage report generated, we can easily integrate it with SonarQube. To add code coverage in SonarQube, we just need to define a few properties for sonar-maven-plugin. To add sonar-maven-plugin for wer project add these lines in <plugins> section of wer pom.xml file:

<plugin>

<groupId>org.sonarsource.scanner.maven</groupId>

<artifactId>sonar-maven-plugin</artifactId>

<version>3.7.0.1746</version>

</plugin>


After including sonar-maven-plugin into wer project we need to define few properties in the pom.xml file. Here are the properties we need to define to integrate code coverage with SonarQube:


<properties>

<java.version>1.8</java.version>

<sonar.language>java</sonar.language>

<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>

<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>

<sonar.coverage.jacoco.xmlReportPaths>${project.build.directory}/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>

<sonar.jacoco.reportsPaths>${project.build.directory}/jacoco-ut.exec</sonar.jacoco.reportsPaths>

<sonar.tests>src/test/java</sonar.tests>

<sonar.projectKey>sonar-test</sonar.projectKey>

<sonar.host.url>http://localhost:9000</sonar.host.url>

</properties>

If we are using Cobertura for generating code coverage report then set it as a value of sonar.java.coveragePlugin property. Make sure the value of sonar.coverage.jacoco.xmlReportsPath property points to the xml code coverage report file generated by jacoco-maven-plugin. If we are using Cobertura then point the sonar.flex.cobertura.reportPaths property to the generated xml report.

Now to push code coverage reports to SonarQube, we need to first generate code coverage reports as part of the build. To generate the report run below the maven goal:

mvn clean install

Once the coverage report is generated, we need to run a sonar plugin for analyzing code by SonarQube by executing the below maven goal:

mvn sonar:sonar -Dsonar.login=<wer-token>

code coverage report on SonarQube:

Automate Code Coverage Reports:- Automate code coverage reports can be generated in bitbucket can be generated with:-

  • Jenkins

  • Sonarcloud