Coverage Quick Start

This guide will help you set up code coverage reporting in your CI pipeline.

Prerequisites

1. A repository with automated tests that generate coverage data
2. A GitHub user with administrator permissions on your organization
3. Ability to modify and run CI scripts for your repository

Step 1. Determine an authentication approach

Qlty supports three ways to authenticate code coverage uploads from your CI system:

1. OpenID Connect (OIDC) with GitHub Actions (recommended)
2. Workspace-level coverage tokens
3. Project-level coverage tokens

If you are using GitHub Actions, we recommend using OIDC as it is the most secure and requires no management of long-lived coverage tokens.

To use OIDC, you will make changes to your GitHub Actions workflow (described in Step 3 below). No other setup is required.

If you are not using GitHub Actions (or can’t use OIDC), you can use a workspace-level coverage token that is scoped to your entire organization.

Obtain a coverage token from Qlty

If you are not using OIDC, you will need to obtain a coverage token:

• Login to Qlty via GitHub OAuth
• Navigate to Workspace Settings > Code Coverage (e.g. https://qlty.sh/gh/your-org/settings/coverage)
• Copy the workspace’s coverage token

Step 2. Store the coverage token as a CI secret (if not using OIDC)

If you are using OIDC, you can skip this step.

A coverage token should be treated as a credential and stored securely within your CI system.

The specifics for how to store a secret at the organization level vary by provider:

• GitHub Actions
• CircleCI
• Buildkite
• Travis CI

Step 3. Update CI workflow scripts

This guide assumes that your CI scripts are already generating code coverage data from the execution of your automated tests. The method to generate this data varies by programming language and testing tools.

Once you have code coverage data being generated in one of our supported formats, continue with the steps below.

GitHub Actions

Add a step to run the qltysh/qlty-action/coverage reusable GitHub Action:

1
# ...
2
permissions:
3
    contents: read
4
    id-token: write # IMPORTANT
5
6
jobs:
7
    build:
8
        # ...
9
        steps:
10
            # ...
11
12
            - uses: qltysh/qlty-action/coverage@v2
13
              with:
14
                  oidc: true
15
                  files: coverage/lcov.info # REPLACE WITH PATH TO YOUR COVERAGE FILE

CircleCI Orb

We provide a CircleCI Orb for uploading coverage reports. Add the following to your .circleci/config.yml file:

1
# ...
2
jobs:
3
    run_tests_and_publish_coverage:
4
        # ...
5
        steps:
6
            # ...
7
            - qlty-orb/coverage_publish:
8
                  files: coverage/lcov.info

Learn more about our CircleCI Orb

CLI Integration / Other CI Providers

If you are not using CircleCI or GitHub, or prefer not to use the convenience wrappers available for those providers, manual integration is straight-forward.

To publish coverage, you will install the Qlty CLI and run the qlty coverage publish command, specifying the coverage data file(s):

$
# Assumes $QLTY_COVERAGE_TOKEN is set in the environment
>
curl https://qlty.sh | sh
>
qlty coverage publish coverage/lcov.info # Replace with path to your coverage data file(s)

Occassionally, qlty coverage publish requires additional arguments. The most commonly used arguments are:

• --add-prefix and/or --strip-prefix to ensure the paths to source files are valid
• --format to specify the format of the coverage file (e.g. --format jacoco)

See our documentation on Manual Integration for more information.

Step 4. Validate coverage uploads from a branch

After updating your CI script:

1. Push your changes to a branch and trigger a build
2. Ensure the build completes successfully with no errors during the upload process
3. On Qlty, navigate to your Project Settings > Code Coverage page
4. Your uploaded coverage report should appear in the table at the bottom of the page
5. Click on the commit SHA to view your report details and verify it matches your expectations

Step 5. Merge your CI script changes into your main branch

Once coverage uploads are working successfully for your branch:

1. Merge the branch into your main branch and run a build
2. Ensure the build completes successfully with no errors from the coverage upload step
3. Validate the coverage upload in your Project Settings
4. Once Qlty receives a valid coverage report for your main branch, your coverage info will be displayed in your project’s Overview page

Optional: Enable coverage commit statuses

To enforce coverage thresholds on pull requests:

• Enable commit statuses (Qlty Coverage and Qlty Diff Coverage) within your Project’s Gate settings
• Adjust coverage check settings (including configurations for minimum diff size and total coverage decrease)

Next Steps

• Generating Coverage Data - Format-specific configuration
• Coverage Metrics - Learn about the coverage metrics Qlty provides
• Coverage Merging - Merge multiple coverage reports from parallel test runs
• Example Repositories - Reference implementations
• Troubleshooting - Common issues and solutions