Generating Coverage Data

To use Qlty’s code coverage features, you first need to generate coverage reports from your test suite. This page provides instructions for common languages and frameworks.

Supported Languages and Formats

Language	Format / Coverage Tool	File Extension
Ruby	Simplecov	`.json`
JavaScript	Lcov (generated by Istanbul)	`.lcov`, `.info`
Python	Cobertura (generated by pytest or coverage.py)	`.xml`
PHP	Clover	`*.xml`
Go	Clover	`*.xml`
Java	Cobertura and Jacoco	`*.xml`
Swift	Cobertura (using native Xcode coverage and slather)	`*.xml`
Kotlin	Cobertura and Jacoco	`*.xml`
Scala	Cobertura (generated by sbt-scoverage)	`*.xml`
C	Lcov (generated by gcov / lcov)	`*.info`
C++	Lcov (generated by gcov / lcov)	`*.info`
.NET	dotCover	`*.xml`
VB.NET	dotCover	`*.xml`
Elixir	Lcov (Generated by lcov_ex)	`*.info`
Qlty	This is Qlty’s generic coverage format	`*.jsonl`

Language-specific Instructions

JavaScript / TypeScript

JavaScript / TypeScript with Jest

Jest has built-in coverage reporting capabilities using Istanbul under the hood.

# Add to your package.json or run directly
"scripts": {
  "test": "jest --coverage"
}

By default, Jest will output coverage reports to the coverage directory, including an LCOV report at coverage/lcov.info.

React

If you’re using Create React App, Jest is already configured. Simply run:

npm test -- --coverage

Node.js with NYC (Istanbul)

# Install NYC
npm install --save-dev nyc

# Add to your package.json
"scripts": {
  "test": "nyc mocha"
}

NYC will generate an LCOV report at coverage/lcov.info.

Python

Python with pytest

# Install pytest-cov
pip install pytest-cov

# Run tests with coverage
pytest --cov=. --cov-report=xml

This will generate a coverage report in Cobertura XML format at coverage.xml.

Python with coverage.py

# Install coverage.py
pip install coverage

# Run your tests with coverage
coverage run -m pytest

# Generate an XML report
coverage xml

This will also generate a coverage report in Cobertura XML format at coverage.xml.

Ruby

Ruby with SimpleCov

Add SimpleCov to your Gemfile:

group :test do
  gem 'simplecov', require: false
end

Configure SimpleCov at the top of your test helper file (e.g., test/test_helper.rb or spec/spec_helper.rb):

require 'simplecov'
SimpleCov.start

# Rest of your test helper file...

Run your tests normally:

bundle exec rspec
# or
bundle exec rails test

SimpleCov will generate a coverage report at coverage/.resultset.json.

Go

Go has built-in coverage capabilities:

# Run tests with coverage
go test -coverprofile=coverage.out ./...

# Convert to Clover format for Qlty using goclover
# First install goclover: go install github.com/t-yuki/goclover@latest
goclover coverage.out > clover.xml

This will generate a Clover XML report which Qlty can understand.

Java / Kotlin

Java / Kotlin with Gradle and JaCoCo

Add JaCoCo to your Gradle build:

plugins {
    id 'jacoco'
}

jacoco {
    toolVersion = "0.8.8"
}

jacocoTestReport {
    reports {
        xml.required = true
        csv.required = false
        html.required = true
    }
}

Run tests and generate a JaCoCo report:

./gradlew test jacocoTestReport

This will generate an XML report at build/reports/jacoco/test/jacocoTestReport.xml.

Java / Kotlin with Maven and JaCoCo

Add JaCoCo to your Maven pom.xml:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.8</version>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>test</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Run tests and generate a JaCoCo report:

mvn clean test

This will generate an XML report at target/site/jacoco/jacoco.xml.

PHP

PHP with PHPUnit

Configure PHPUnit in your phpunit.xml:

<phpunit>
    <!-- ... other configuration ... -->
    <coverage>
        <report>
            <clover outputFile="coverage.xml"/>
        </report>
    </coverage>
</phpunit>

Run PHPUnit:

phpunit --coverage-clover=coverage.xml

This will generate a Clover XML report at coverage.xml.

Swift

Swift Options

While there are multiple extraction options for Swift coverage data, each involves the same initial setup step: enable code coverage generation on your Xcode project or build. This can be done in your Xcode scheme, or via the xcodebuild command line option enableCodeCoverage YES.By providing these settings, Xcode will generate coverage data into a binary format. Coverage data next needs to be exported into a format recognized by Qlty.If you’re using Xcode 9.3 or later (recommended for most modern projects), we recommend using xccov to export coverage data, but you can also use Slather.If using an older version of Xcode, Slather is likely your best option.

Using `xccov` (Xcode >= 9.3)

xccov was introduced in Xcode 9.3 as a simplified way to extract coverage data. xccov enables extracting coverage data from a single consolidated .xcresult bundle, greatly simplifying coverage setup. To use xccov:

Find / specify the location of the *.xcresult bundle If you are writing an Xcode build command yourself, you can supply it with the location where you’d like it to write the *.xcresult:
xcodebuild test -scheme MyTestProject -destination 'platform=macOS' -enableCodeCoverage YES -resultBundlePath ./TestResults.xcresult
If instead you are utilizing a CI provider’s Xcode test step which then executes Xcode on your behalf, examine the underlying xcodebuild command it uses. It may provide a standardized location for the .xcresult bundle. If not, the step may export an environment variable containing the path to the .xcresult bundle.
Export coverage data using xccov

Qlty supports the xccov format which is output when passed --archive and --json are passed as arguments as follows:

xcrun xccov view --archive --json TestResults.xcresult > xccov-coverage.json

xccov supports two (poorly documented) coverage export JSON schemas generated with the command line arguments --archive and --report. The --archive format is the only supported format, as it’s the only format that contains information about which specific lines were executed and which were not. Below are examples of the two formats for reference.

{
    "/Users/ubuntu/code/qltysh/example-swift/Sources/MyTestProject/MyTestProject.swift": [
        {
        "isExecutable": false,
        "line": 1
        },
        {
        "executionCount": 5,
        "isExecutable": true,
        "line": 2
        },
        {
        "isExecutable": false,
        "line": 3
        },
        {
        "executionCount": 3,
        "isExecutable": true,
        "line": 4
        }
    ]
}

{
    "coveredLines": 78,
    "executableLines": 78,
    "lineCoverage": 1,
    "targets": [
        {
        "buildProductPath": "/Users/ubuntu/Library/Developer/Xcode/DerivedData/example-swift-hchpojreepctqnbgwylrabwofawj/Build/Products/Debug/PackageFrameworks/MyTestProject_6D367B2C6E48974A_PackageProduct.framework/MyTestProject_6D367B2C6E48974A_PackageProduct",
        "coveredLines": 14,
        "executableLines": 14,
        "files": [
            {
            "coveredLines": 14,
            "executableLines": 14,
            "functions": [
                {
                "coveredLines": 1,
                "executableLines": 1,
                "executionCount": 5,
                "lineCoverage": 1,
                "lineNumber": 2,
                "name": "Calculator.init()"
                }
            ],
            "lineCoverage": 1,
            "name": "MyTestProject.swift",
            "path": "/Users/ubuntu/code/qltysh/example-swift/Sources/MyTestProject/MyTestProject.swift"
            }
        ],
        "lineCoverage": 1,
        "name": "MyTestProject"
        }
    ]
}

When uploading this coverage data to Qlty, specify the format of the file as “xccov-json”

For example, for the GitHub coverage action you would specify:

- name: Upload coverage report
  uses: qltysh/qlty-action/coverage@ea1eaf434a27bf50cd544153084fbb11c52aaf84
  with:
    oidc: true
    format: xccov-json
    files: xccov-coverage.json

Swift with Xcode and Slather (see Slather’s documentation for supported Xcode versions)

Slather is a Ruby gem that extracts coverage data from Xcode projects.On the pro side, Slather supports a wide range of Xcode versions, supports excluding files from coverage, and is fairly easy to use.On the con side, Slather relies on lower level Apple tools like llvm-cov which is more complex and relies on being able to locate coverage-related files. On simple projects this may be fine. More complex multi-module projects need extra setup to ensure all coverage data is extracted. See Slather’s documentation for more information. In general the setup process is as follows:

Install Slather
gem install slather

Create a .slather.yml file:

coverage_service: cobertura_xml
xcodeproj: YourProject.xcodeproj
scheme: YourScheme
output_directory: coverage

Run your tests and then generate a coverage report:

xcodebuild -project YourProject.xcodeproj -scheme YourScheme -derivedDataPath Build/ -enableCodeCoverage YES test
slather

This will generate a Cobertura XML report in the coverage directory.

When uploading coverage, specify the Cobertura format and provide the file generated by slather

Elixir

Elixir with lcov_ex

Add lcov_ex to your dependencies in mix.exs:

defp deps do
  [
    {:lcov_ex, "~> 0.3", only: [:dev, :test], runtime: false}
  ]
end

Install the dependency:

mix deps.get

Option 1: Using the mix lcov task

Run the lcov task to run tests and generate coverage:

mix lcov

This will generate a coverage report at cover/lcov.info.To output to a different folder:

mix lcov --output coverage

This will generate the report at coverage/lcov.info.

Option 2: Using mix test —cover

Alternatively, configure LcovEx as your test coverage tool in mix.exs:

def project do
  [
    app: :your_app,
    version: "0.1.0",
    elixir: "~> 1.15",
    deps: deps(),
    test_coverage: [tool: LcovEx, output: "coverage"]
  ]
end

Then run tests with coverage:

mix test --cover

This will generate a coverage report at coverage/lcov.info.

Umbrella Projects

For umbrella projects, running mix lcov at the umbrella level will generate coverage reports for all individual apps and compile them into a single file at cover/lcov.info.To keep the individual app coverage files as well:

mix lcov --keep

Path Fixing

For Qlty to accurately match a coverage report file to files in your Git repository, paths must align. The paths in your coverage report should be relative to your Git project’s root, but this isn’t always the case by default. To resolve path mismatches, the qlty coverage publish command supports these flags:

--add-prefix: Adds a prefix to file paths in coverage payloads
--strip-prefix: Removes a prefix from absolute paths in coverage payloads

Example:

# If your coverage report has absolute paths like /home/user/project/src/file.js
# but should be relative like src/file.js
qlty coverage publish --strip-prefix=/home/user/project coverage.xml

Custom Formats with the Qlty Format

In the unlikely event that your coverage tool does not export coverage data in a supported format, you can convert your coverage data to the qlty format. Learn more about the Qlty format

Next Steps

Once you’ve generated a coverage report, you can:

Upload it to Qlty
Set up CI integration to automate uploads
Merge multiple reports if you have parallel tests

Getting Started

Coverage In Your Workflow

Concepts

Features

Advanced Configuration

Reference

Generating Coverage Data

Supported Languages and Formats

Language-specific Instructions

JavaScript / TypeScript with Jest

React

Node.js with NYC (Istanbul)

Python with pytest

Python with coverage.py

Ruby with SimpleCov

Go

Java / Kotlin with Gradle and JaCoCo

Java / Kotlin with Maven and JaCoCo

PHP with PHPUnit

Swift Options

Using `xccov` (Xcode >= 9.3)

Swift with Xcode and Slather (see Slather’s documentation for supported Xcode versions)

Elixir with lcov_ex

Option 1: Using the mix lcov task

Option 2: Using mix test —cover

Umbrella Projects

Path Fixing

Custom Formats with the Qlty Format

Next Steps

​Supported Languages and Formats

​Language-specific Instructions

​JavaScript / TypeScript with Jest

​React

​Node.js with NYC (Istanbul)

​Python with pytest

​Python with coverage.py

​Ruby with SimpleCov

​Go

​Java / Kotlin with Gradle and JaCoCo

​Java / Kotlin with Maven and JaCoCo

​PHP with PHPUnit

​Swift Options

​Using xccov (Xcode >= 9.3)

​Swift with Xcode and Slather (see Slather’s documentation for supported Xcode versions)

​Elixir with lcov_ex

​Option 1: Using the mix lcov task

​Option 2: Using mix test —cover

​Umbrella Projects

​Path Fixing

​Custom Formats with the Qlty Format

​Next Steps

Supported Languages and Formats

Language-specific Instructions

JavaScript / TypeScript with Jest

React

Node.js with NYC (Istanbul)

Python with pytest

Python with coverage.py

Ruby with SimpleCov

Go

Java / Kotlin with Gradle and JaCoCo

Java / Kotlin with Maven and JaCoCo

PHP with PHPUnit

Swift Options

Using `xccov` (Xcode >= 9.3)

Swift with Xcode and Slather (see Slather’s documentation for supported Xcode versions)

Elixir with lcov_ex

Option 1: Using the mix lcov task

Option 2: Using mix test —cover

Umbrella Projects

Path Fixing

Custom Formats with the Qlty Format

Next Steps