Carry Forward Tags
Carry forward tags allow Qlty to automatically reuse coverage data from previous commits when selective testing leaves coverage gaps, ensuring your coverage metrics remain complete and accurate even when only running a subset of tests.
Background
Many modern CI systems are configured to only run a subset of tests on each commit (also known as “selective testing”). One example of selective testing are “monorepos” where each folder represents a distinct component (or service). These repositories are often configured to limit test runs to only components that have changed.
With selective testing, coverage cannot be strictly compared across commits because those commits may have run different test suites, generating coverage reports that are unrelated. Because coverage data is associated with commits, selective testing creates unique challenges associated with per-commit coverage:
- The latest commit (alone) for a project won’t have comprehensive coverage data
- A pull request is a comparison across 2 commits, and these commits can have different coverage reports
Qlty solves this problem with the intelligent use of coverage tags: if a project defines a tag per component, Qlty “carries forward” coverage for a tag from a previous commit onto a more recent relevant commit. While this behavior is seamless and requires no setup other than defining tags, understanding Carry Forward Tags can help you comprehend how coverage calculations are performed in selective testing scenarios.
How Carry Forward Works
When Qlty processes coverage for a commit, it automatically fills in missing tag data by carrying forward the most recent coverage report for each tag.
For example, in the following diagram, while the most recent commit, at the top, does not have direct coverage report data, the coverage data associated with the coverage tags “component A” and “component B” will be carried forward.
This ensures that your dashboards, PR checks, and coverage overlays always show the most complete and up-to-date coverage information available, even if that coverage information spans multiple commits.
Best Practices
To make the most of the automatic carry forward behavior:
Consistent Tag Usage
Use consistent tag names across all your coverage uploads. This ensures proper tracking and carry-forward behavior.
Regular Complete Coverage
Periodically run and upload coverage for all tags, such as on your main branch or nightly builds. This ensures carried forward data doesn’t become too outdated.
Limitations
While carry forward is a valuable built-in behavior, it’s important to understand its limitations:
- Accuracy over time: Carried-forward data may become outdated if a component goes untested for a long period
- File-level granularity: Carry forward operates at the tag level, not at individual file or line level
- Branch integrity: Data is only carried forward within the same branch history
- Tag consistency: Carry-forward requires consistent usage of tags across commits — e.g. Carry forward will not work properly if the same tag reports information for a different set of tests each time
See Also
- Coverage Tags - How to organize coverage with tags
- Coverage Metrics - Understanding coverage measurements