Coverage Merging

Many builds generate multiple code coverage reports. Some examples include:

• Builds involving multiple programming languages, where each language requires different coverage instrumentation
• Parallelized builds where each parallel unit (thread, process, virtual machine etc.) may generate its own coverage report
• Different test suites (unit, integration, end-to-end) generating separate reports

Qlty supports merging these separate reports into a single comprehensive report using either Client-Side or Server-Side merging.

• If builds occur on a single machine (or multiple machines with shared storage), client-side merging is often the simplest approach.
• If builds are spread across multiple machines and disks, server-side merging is likely the easiest solution.

Client-side merging

Client-side merging uses the Qlty CLI to combine multiple coverage reports into a single report. That single report is then uploaded to Qlty Cloud for processing.

This is accomplished by passing the locations of your raw coverage reports (as paths or globs) to the qlty coverage publish subcommand:

1
- uses: qltysh/qlty-action/coverage@v1
2
  with:
3
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
4
    files: |
5
        coverage/coverage1.lcov
6
        coverage/coverage2.lcov
7
        coverage/coverage3.lcov

Or using glob patterns:

1
- uses: qltysh/qlty-action/coverage@v1
2
  with:
3
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
4
    files: coverage/*.lcov

The CLI transforms each of these reports into the Qlty format, then publishes a single coverage report to Qlty Cloud.

Client-side merging is ideal when all of your coverage data files are on a single machine.

Server-side merging

With server-side merging, a build publishes a series of partial reports to Qlty; when Qlty Cloud knows it’s received all parts (as determined by the below options), it aggregates and reports coverage for that commit SHA.

Server-side merging is ideal when:

• Your tests run on multiple machines without shared storage
• You want to avoid collecting all reports in a single location
• Each report file may require unique path fixing

Qlty provides two strategies for server-side merging:

total-parts-count

This approach works by specifying a --total-parts-count=NUM argument to the qlty coverage publish subcommand when publishing a coverage part. If, for example, your build consists of 2 parallelized builds, Qlty Cloud will wait for 2 coverage publish commands:

1
- uses: qltysh/qlty-action/coverage@v1
2
  with:
3
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
4
    files: coverage/coverage1.lcov
5
    total-parts-count: 2
6
7
- uses: qltysh/qlty-action/coverage@v1
8
  with:
9
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
10
    files: coverage/coverage2.lcov
11
    total-parts-count: 2

Use total-parts-count when you can easily know exactly how many coverage uploads will be made.

coverage complete

This approach is useful when you don’t know the total number of parts in advance, or in complex CI environments where determining the total number ahead of time is impractical.

1. Upload all parts with the --incomplete flag:

1
- uses: qltysh/qlty-action/coverage@v1
2
  with:
3
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
4
    files: coverage/coverage1.lcov
5
    incomplete: true
6
7
- uses: qltysh/qlty-action/coverage@v1
8
  with:
9
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
10
    files: coverage/coverage2.lcov
11
    incomplete: true

2. After all parts have been uploaded, send a “Complete” signal to tell Qlty that all parts have been received:

$
# Tell Qlty all parts have been uploaded
>
qlty coverage complete

When Qlty receives the completion signal, it will finalize the coverage report, merging all the received parts.

Use the coverage complete command when:

• The number of parts isn’t easily known everywhere in advance
• Your CI workflow has dynamic job generation
• You have a complex build pipeline where tracking total parts is difficult

Client and server-side merging together

Client-side and server-side merging can also be used together: each machine itself can combine multiple coverage reports using client-side merging, and then post that combined report to Qlty as part of server-side merging.

The total parts count always equals the number of total publish commands executed, regardless of the number of paths passed to the subcommand. If 2 machines perform client-side aggregation of N paths, qlty coverage publish should specify “2” as the total parts count.

1
# Job 1: Combine and upload unit test reports
2
- uses: qltysh/qlty-action/coverage@v1
3
  with:
4
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
5
    total-parts-count: 2
6
    files: |
7
        unit1.lcov
8
        unit2.lcov
9
        unit3.lcov
10
11
# Job 2: Combine and upload integration test reports
12
- uses: qltysh/qlty-action/coverage@v1
13
  with:
14
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
15
    total-parts-count: 2
16
    files: |
17
        integration1.lcov
18
        integration2.lcov

Merging with coverage tags

The qlty coverage publish command, whether used for client or server-side merging, is always for a single tag (no no tag).

Client-side merging with tags

With client-side merging, you need to publish N reports if there are N tags (If there are 3 coverage tags, the build should execute qlty coverage publish 3 times):

1
# Publish unit test coverage (combining multiple reports)
2
- uses: qltysh/qlty-action/coverage@v1
3
  with:
4
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
5
    tag: unit
6
    files: |
7
        unit1.lcov
8
        unit2.lcov
9
10
# Publish integration test coverage (combining multiple reports)
11
- uses: qltysh/qlty-action/coverage@v1
12
  with:
13
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
14
    tag: integration
15
    files: |
16
        integration1.lcov
17
        integration2.lcov

Server-side merging with tags

total-parts-count with tags

When using the total-parts-count approach with tags, you need to specify the total number of parts for each tag separately. This means that if you have multiple tags, you should publish each tag’s parts separately.

With the total-parts-count approach, the --total-parts-count should reflect the number of total parts Qlty Cloud should expect for that tag.

For example, if there’s 4 reports with an “integration” tag and 2 reports with a “unit” tag, your build should publish 4 integration reports and 2 unit reports as follows:

1
# Integration tag example (1 of 4)
2
- uses: qltysh/qlty-action/coverage@v1
3
  with:
4
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
5
    tag: integration
6
    total-parts-count: 4
7
    files: coverage.lcov
8
# ... Three more `qltysh/qlty-action/coverage` steps for the `integration` tag ...
9
10
# Unit tag example (1 of 2)
11
- uses: qltysh/qlty-action/coverage@v1
12
  with:
13
    token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
14
    tag: unit
15
    total-parts-count: 2
16
    files: coverage.lcov
17
# ... One more `qltysh/qlty-action/coverage` step for the `unit` tag ...

coverage complete with tags

When using the --incomplete flag approach with tags, you must complete each tag separately. For each tag, upload all parts with the --incomplete flag, then send a completion signal for that specific tag:

$
qlty coverage publish --tag=unit --incomplete unit1.lcov
>
qlty coverage publish --tag=unit --incomplete unit2.lcov
>
qlty coverage complete --tag=unit
>
>
qlty coverage publish --tag=integration --incomplete coverage1.lcov
>
qlty coverage publish --tag=integration --incomplete coverage2.lcov
>
qlty coverage publish --tag=integration --incomplete coverage3.lcov
>
qlty coverage complete --tag=integration

Common Scenarios

Build matrix

In GitHub Actions or other CI systems that support matrix builds:

1
jobs:
2
  test:
3
    strategy:
4
      matrix:
5
        node-version: [14.x, 16.x, 18.x]
6
        os: [ubuntu-latest, windows-latest]
7
8
    runs-on: ${{ matrix.os }}
9
10
    steps:
11
      # ... run tests and generate coverage ...
12
13
      - name: Upload coverage
14
        uses: qltysh/qlty-action/coverage@v1
15
        with:
16
          token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
17
          files: coverage/lcov.info
18
          # 3 node versions × 2 operating systems = 6 parts
19
          total-parts-count: 6

Multiple languages

For a project with both JavaScript and Python code:

1
jobs:
2
  test-javascript:
3
    runs-on: ubuntu-latest
4
    steps:
5
      # ... run JS tests and generate coverage ...
6
7
      - name: Upload JS coverage
8
        uses: qltysh/qlty-action/coverage@v1
9
        with:
10
          token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
11
          files: coverage/lcov.info
12
          total-parts-count: 2 # JavaScript + Python = 2 parts
13
14
  test-python:
15
    runs-on: ubuntu-latest
16
    steps:
17
      # ... run Python tests and generate coverage ...
18
19
      - name: Upload Python coverage
20
        uses: qltysh/qlty-action/coverage@v1
21
        with:
22
          token: ${{ secrets.QLTY_COVERAGE_TOKEN }}
23
          files: coverage.xml
24
          total-parts-count: 2 # JavaScript + Python = 2 parts

Verifying merging

After publishing your coverage reports, you can verify that merging was successful:

1. Navigate to your project in Qlty Cloud
2. Go to Project Settings → Code Coverage
3. Filter the table below “Code Coverage Setup” by commit
4. You should see N + 1 reports:
   • N individual parts representing the reports you sent
   • 1 final merged report representing the combined coverage

Troubleshooting

Missing parts

If your merged report doesn’t appear after several minutes:

For Total Parts Count approach:

1. Check that all parts were successfully uploaded
2. Verify that the --total-parts-count value matches the actual number of uploads
3. Ensure all parts were uploaded for the same commit SHA and branch
4. Check that all parts used the same tag (if using tags)

Completion Signal Issues

If you’re using the incomplete flag approach and experiencing issues:

1. Ensure that all parts were uploaded with the --incomplete flag (otherwise, qlty coverage publish assumes by default that the report is complete)
2. Verify the completion command was executed for each tag you’re tracking
3. Check that both upload and completion commands are using the same credentials and workspace
4. Verify all coverage parts were uploaded with the same commit SHA and branch

Partial merging

If some files are missing from your merged coverage:

1. Verify that all coverage reports were valid and complete
2. Check for path differences between reports
3. Ensure all reports use compatible formats
4. Consider using path fixing options if paths don’t match

See Also

• Coverage Tags - Track different types of tests separately
• CI Integration - Automate coverage reporting in your CI pipeline
• Path Fixing - Fix path discrepancies in coverage reports