Concepts

CI Integration / Uploader

Integrating code coverage reporting into your CI pipeline ensures that coverage data is consistently collected and reported for every build.

Qlty is compatible with every major CI provider, and offers specific workflow support for GitHub (via a GitHub Action) and Circle CI (via an Orb).

CI Provider Integration

GitHub Actions

Use Qlty’s official GitHub Action for the simplest integration.

1# .github/workflows/test.yml
2name: Test and Coverage
3
4on:
5  push:
6    branches: [ main ]
7  pull_request:
8    branches: [ main ]
9
10permissions:
11  contents: read
12  id-token: write # Required for OIDC
13
14jobs:
15  test:
16    runs-on: ubuntu-latest
17    steps:
18      - uses: actions/checkout@v3
19      
20      # Run your tests with coverage
21      - name: Run tests
22        run: npm test -- --coverage
23      
24      # Upload coverage to Qlty
25      - uses: qltysh/qlty-action/coverage@v2
26        with:
27          oidc: true
28          files: coverage/lcov.info

Using a Coverage Token

1# .github/workflows/test.yml
2name: Test and Coverage
3
4on:
5  push:
6    branches: [ main ]
7  pull_request:
8    branches: [ main ]
9
10jobs:
11  test:
12    runs-on: ubuntu-latest
13    steps:
14      - uses: actions/checkout@v3
15      
16      # Run your tests with coverage
17      - name: Run tests
18        run: npm test -- --coverage
19      
20      # Upload coverage to Qlty
21      - uses: qltysh/qlty-action/coverage@v2
22        with:
23          token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
24          files: coverage/lcov.info

CircleCI

Qlty offers a CircleCI Orb for easy integration.

Using the Orb

Add the Qlty orb to your .circleci/config.yml:

1version: 2.1
2
3orbs:
4  qlty-orb: qltysh/qlty-orb@0.0
5
6jobs:
7  test:
8    docker:
9      - image: cimg/node:16.13
10    steps:
11      # Checkout is required
12      - checkout
13      - run:
14          name: Install dependencies
15          command: npm ci
16      - run:
17          name: Run tests with coverage
18          command: npm test
19      - qlty-orb/coverage_publish:
20          files: coverage/lcov.info
21
22workflows:
23  main:
24    jobs:
25      - test

Manual Integration

If you prefer not to use the orb:

1version: 2.1
2
3jobs:
4  test:
5    docker:
6      - image: cimg/node:16.13
7    steps:
8      - checkout
9      - run:
10          name: Install dependencies
11          command: npm ci
12      - run:
13          name: Run tests with coverage
14          command: npm test
15      - run:
16          name: Install Qlty CLI
17          command: curl https://qlty.sh | sh
18      - run:
19          name: Upload coverage
20          command: qlty coverage publish coverage/lcov.info
21          environment:
22            QLTY_COVERAGE_TOKEN: ${QLTY_COVERAGE_TOKEN}
23
24workflows:
25  main:
26    jobs:
27      - test

Travis CI

Add Qlty coverage reporting to your .travis.yml:

1language: node_js
2node_js:
3  - 16
4
5script:
6  - npm test -- --coverage
7
8after_success:
9  - curl https://qlty.sh | sh
10  - qlty coverage publish coverage/lcov.info

Make sure to add your QLTY_COVERAGE_TOKEN in the Travis CI repository settings.

GitLab CI

Add coverage reporting to your .gitlab-ci.yml:

1test:
2  stage: test
3  script:
4    - npm ci
5    - npm test -- --coverage
6    - curl https://qlty.sh | sh
7    - qlty coverage publish coverage/lcov.info
8  artifacts:
9    paths:
10      - coverage/

Add your QLTY_COVERAGE_TOKEN as a CI/CD variable in your GitLab project settings.

Buildkite

Add the following steps to your pipeline.yml:

1steps:
2  - label: "Run tests with coverage"
3    commands:
4      - npm ci
5      - npm test -- --coverage
6      
7  - label: "Upload coverage to Qlty"
8    commands:
9      - curl https://qlty.sh | sh
10      - qlty coverage publish coverage/lcov.info
11    env:
12      QLTY_COVERAGE_TOKEN: "${QLTY_COVERAGE_TOKEN}"

Manual Integration

The Qlty CLI coverage sub-commands leverage as much information from its environment as possible to ease integration. This includes environment variables as well as git metadata from the current working directory. However, sometimes Qlty is run without access to one or both of these. In these cases, customers must supply command line arguments to coverage publish to replace what otherwise would come from the environment. These are referred to as “overrides” because they also override the values, if any, detected from the environment.

Which of these are necessary depends on your environment — when running qlty coverage publish, the command will error with which overrides are required if it’s unable to detect them from the environment.

Reference Overrides

When publishing coverage, Qlty associates the coverage data with a single reference. The reference can be one of:

  • a branch
  • a pull request
  • a git tag

When running in a CI workflow, qlty is often able to determine the reference from environment variables provided by the CI vendor. Without this, users must provide one of the following to help Qlty construct the appropriate reference:

  • pull request: --override-pr-number <PR_NUMBER>
  • branch: --override-branch <BRANCH_NAME> (can be used alongside --override-pr-number)
  • git tag: --override-git-tag <TAG_NAME> (cannot be combined with either --override-pr-number or --override-branch)

Commit SHA Override

A commit sha is always required when sending coverage data to Qlty.sh. Similarly, this can be manually specified with:

--override-commit-sha <COMMIT_SHA>

Commit Time Override

A timestamp associated with the commit is also required. This can be specified with:

--override-commit-time <OVERRIDE_COMMIT_TIME>

The commit time can be a Unix timestamp (seconds since epoch) or a date in RFC3339/ISO8601 format.

Examples

$QLTY_COVERAGE_TOKEN=QLTY_TOKEN qlty coverage publish --override-branch=main --override-commit-sha=e83c5163316f89bfbde7d9ab23ca2e25604af290 --override-commit=time=1704067200 lcov.info

Advanced CI Configuration

Parallelized Tests

When running tests in parallel, you’ll need to merge the coverage reports before uploading. Qlty supports both client-side and server-side merging.

Client-side Merging

Collect all coverage reports into a single location and upload them together:

$qlty coverage publish report1.lcov report2.lcov report3.lcov

Server-side Merging

Upload each part separately and specify the total number of parts:

$# Part 1
>qlty coverage publish --total-parts-count=3 report1.lcov
>
># Part 2
>qlty coverage publish --total-parts-count=3 report2.lcov
>
># Part 3
>qlty coverage publish --total-parts-count=3 report3.lcov

Learn more about Coverage Merging.

Multiple Test Suites

If you have different types of tests (unit, integration, e2e), you can track them separately using coverage tags:

$# Upload unit test coverage
>qlty coverage publish --tag=unit unit-coverage.lcov
>
># Upload integration test coverage
>qlty coverage publish --tag=integration integration-coverage.lcov

Learn more about Coverage Tags.

CI Provider Detection

Qlty automatically detects and extracts metadata from these CI providers:

  • GitHub Actions
  • CircleCI
  • Travis CI
  • GitLab CI
  • Buildkite
  • Codefresh
  • Semaphore

If your CI provider is not listed above, double check our Qlty repository to see if an implementation has been added recently (and let us know so we can update our docs).

If your CI provider is not support, you’ll need to manually provide the following required arguments to qlty coverage publish:

  • --override-branch
  • --override-commit
  • --override-build-id

Troubleshooting

Common Issues

  • Authentication failures: Check that your token is correctly set as an environment variable
  • Report format not recognized: Make sure you’re using a supported coverage format
  • Path mismatches: Use the --strip-prefix and --add-prefix options to fix file paths
  • Missing metadata: Use --override-branch, --override-commit and --override-build-id if CI detection fails

See Also