In recent posts on code coverage, Alan discussed how to Enable Code Coverage in JCL and How to Capture Code Coverage without Starting IDz. In this post, I will show how to use the Headless Code Coverage collector to export in SonarQube format and display the code coverage result in SonarQube dashboard.

Installing the Headless Code Coverage Collector

To use this function, make sure to select the optional Headless Code Coverage for IBM z/OS Debugger feature when installing IDz. This will result in a headless-cc folder being installed.

Exporting code coverage data using Exportertype

Starting in IDz 14.1.2 (announced as part of IBM Application Delivery Foundation for z Systems V3.1.1),  you can use the  new exportertype option to export code coverage data into different formats. You can specify exporting to multiple exporter types by separating them by a comma. The supported exporter types are:

  • CCRESULT – generates the compressed format with the .cczip extension
  • CCSONARQUBE – generates the SonarQube format with the .xml extension
  • CCRAW – generates the raw format with the .zip extension

For example, run the following command in the headless-cc folder to start the daemon on port 8006 and to export to all three exporter type:

        codecov –startdaemon –port=8006 -exportertype=ccresult,ccsonarqube,ccraw

This new exportertype option enables you to export the code coverage data in a format that can be imported by other tools.

Exporting to SonarQube format

You can run the code coverage headless collector with exportertype set to ccsonarqube to export the result in SonarQube format. You should also specify the savesource option to save the source with the code coverage data. For example,

        codecov –startdaemon –port=8006 –savesource -exportertype=ccsonarqube

After you start the daemon, run the JCL with settings that will connect to the headless code coverage collector. This include setting the machine specific TCPIP address, port number and specifying EQA_STARTUP_KEY=CC. Refer to steps in Enable Code Coverage in JCL and How to Capture Code Coverage without Starting IDz for more details.

After you submitted the JCL with code coverage enabled, the exported XML file and source will be saved on the machine running the headless collector.

 

This is an example of the generated xml file with code coverage data in SonarQube format:

You can follow the instructions in SonarQube documentation Analyzing with SonarQube Scanner to setup the scanner that will import code coverage data into SonarQube. You can then see the code coverage result in the SonarQube dashboard.

 

Importing code coverage result into SonarQube

For the SonarQube Scanner to find and import the code coverage data, copy the exported code coverage XML file and the saved source into a directory (called ATMIMS in this example).

Then modify the sonar-project.properties file to point to the source directory and the XML file.

Next, run SonarQube Scanner to scan the code coverage data into SonarQube.

For example,

        sonar-scanner.bat –D’sonar.projectBaseDir’=ATMIMS

 

Displaying imported result in SonarQube dashboard

After running SonarQube Scanner, paste the project URL in a browser to bring up the SonarQube project dashboard.

Click on the coverage percentage on the project dashboard to get detail breakdown of code coverage by source file.

Click on a source file to bring up the source listing and see the coverage information right in the source. Lines that are covered are indicated by a green ruler while lines that are not covered are indicated by a red ruler.

It’s easy to export code coverage data

As you can see, it is really easy to set up a code coverage headless collector without a UI so that it can be run automatically, for example, as part of the build process. The code coverage result can be exported to different formats so that it can be consumed by different tools such as SonarQube. You can then fully utilize the capability of these tools to analyze the code coverage data.

3 comments on"Importing IDz Code Coverage Result into SonarQube"

  1. Hi, I’ve tried this feature and found it is still a lot of gaps in between, before we can consume this feature smoothly:

    1. The source from the code coverage data is different from the original source, it has all copybook expanded but still keep the COPY statement.

    2. Usually the source in the codecoverage data xml are shown as PDS name, e.g. “xxxx.pds.listing(sam1).cob”. This will flag an issue in SonarQube as the file name is different from the program name. This induce a lot of manual work to rename this file before it can be uploaded to Sonar.

    3. Even that, the number of issues and other metrics are different from the actual program source. This make it difficult for users to manage the SonarQube issue.

    4. If we still upload the actual source and ignore the source in the codecoverage data, then the codecoverage line number will not be matched.

    Any suggestion?

    • I believe it will help if I give you some background information.

      Code coverage technology relies on the debug information generated by the compiler and consumed by the debugger.
      In the case of COBOL, it is only recently (v6.x) that the compiler can produce debug info that points to the source file (v6.2 is required for code coverage).
      Prior to that the compiler only generated an expanded listing and all debugging and code coverage was mapped to the listing.
      That is why the code coverage collector (the part that runs the program and collects code coverage) supports packaging the expanded listing with the cc data (the -savesource option) so that you always have something that matches the cc data. In the example given above the “source” is actually the expanded listing. I realize the use of the term source is a bit misleading but the same code coverage technology is used by many different platforms and languages so it uses the more generic term “source”.

      This background information explains why you are having the problems identified in your comments (1,3,4)

      The good news is that COBOL v6.2 has support to generate code coverage data that points to the source file.
      I will make another append when I have more information on the steps required to enable source level code coverage.

      Your comment #2 brings up in interesting point that I’ll have to investigate and that is the names we generate for the source/listing.
      For the import into SonarQube to go smoothly, the names have to match the source that you already have imported into SonarQube.
      I’m going to look at the names we store in the cc data when source level information from the compiler is being used.

      Thank you for your feedback on this feature.

      • Hi Alan, I wonder if there is some updates on your investigate on my previously comment #2, or any other news for this integration?

Join The Discussion

Your email address will not be published. Required fields are marked *