Getting Started

Coverage Quick Start

This guide will help you set up code coverage reporting in your CI pipeline.

Prerequisites

  1. A repository with automated tests that generate coverage data
  2. A GitHub user with administrator permissions on your organization
  3. Ability to modify and run CI scripts for your repository

Step 1. Determine an authentication approach

Qlty supports three ways to authenticate code coverage uploads from your CI system:

  1. OpenID Connect (OIDC) with GitHub Actions (recommended)
  2. Workspace-level coverage tokens
  3. Project-level coverage tokens

If you are using GitHub Actions and are able to use OIDC, we recommend that approach as it is the most secure and requires no management of long-lived coverage tokens.

If you are not using GitHub Actions, or are unable to use OIDC, you can use a workspace-level coverage token that is scoped to your entire organization.

Obtain a coverage token from Qlty

If you are not using OIDC, you will need to obtain a coverage token:

  • Login to Qlty via GitHub OAuth
  • Navigate to Workspace Settings > Code Coverage (e.g. https://qlty.sh/gh/your-org/settings/coverage)
  • Copy the workspace’s coverage token

Step 2. Store the coverage token as a CI secret (if not using OIDC)

If you are using OIDC, you can skip this step.

A coverage token should be treated as a credential and stored securely within your CI system.

The specifics for how to store a secret at the organization level vary by provider:

Step 3. Update CI workflow scripts

This guide assumes that your CI scripts are already generating code coverage data from the execution of your automated tests. The method to generate this data varies by programming language and testing tools.

Once you have code coverage data being generated in one of our supported formats, continue with the steps below.

GitHub Actions

Add a step to run the qltysh/qlty-action/coverage reusable GitHub Action:

1# ...
2permissions:
3    contents: read
4    id-token: write # IMPORTANT
5
6jobs:
7    build:
8        # ...
9        steps:
10            # ...
11
12            - uses: qltysh/qlty-action/coverage@v1
13              with:
14                  oidc: true
15                  files: COVERAGE_FILE_PATH # e.g. coverage/lcov.info

If you are using a coverage token instead of OIDC, remove the oidc: true line and add token: ${{ secrets.QLTY_COVERAGE_TOKEN }} to the with block:

1# ...
2jobs:
3    build:
4        # ...
5        steps:
6            # ...
7
8            - uses: qltysh/qlty-action/coverage@v1
9              with:
10                  token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
11                  files: COVERAGE_FILE_PATH # e.g. coverage/lcov.info

CircleCI Orb

We provide a CircleCI Orb for uploading coverage reports. Add the following to your .circleci/config.yml file:

1# ...
2jobs:
3    run_tests_and_publish_coverage:
4        # ...
5        steps:
6            # ...
7            - qlty/coverage_publish:
8                  files: coverage/lcov.info

Learn more about our CircleCI Orb

Other CI providers

With other CI providers, install the Qlty CLI and run the qlty coverage upload command:

$# Assumes $QLTY_COVERAGE_TOKEN is set in the environment
>curl https://qlty.sh | sh
>qlty coverage publish COVERAGE_FILE_PATH

The Qlty CLI auto-detects workflow metadata from environment variables on the following platforms:

  • Buildkite
  • Travis CI
  • CodeFresh
  • Semaphore

Step 4. Validate coverage uploads from a branch

After updating your CI script:

  1. Push your changes to a branch and trigger a build
  2. Ensure the build completes successfully with no errors during the upload process
  3. On Qlty, navigate to your Project Settings > Code Coverage page
  4. Your uploaded coverage report should appear in the table at the bottom of the page
  5. Click on the commit SHA to view your report details and verify it matches your expectations

Step 5. Merge your CI script changes into your main branch

Once coverage uploads are working successfully for your branch:

  1. Merge the branch into your main branch and run a build
  2. Ensure the build completes successfully with no errors from the coverage upload step
  3. Validate the coverage upload in your Project Settings
  4. Once Qlty receives a valid coverage report for your main branch, your coverage info will be displayed in your project’s Overview page

Optional: Enable coverage commit statuses

To enforce coverage thresholds on pull requests:

  • Enable commit statuses (Qlty Coverage and Qlty Diff Coverage) within your Project’s Gate settings
  • Adjust coverage check settings (including configurations for minimum diff size and total coverage decrease)

Next Steps