Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.qlty.sh/llms.txt

Use this file to discover all available pages before exploring further.

Qlty provides several different code coverage metrics to give you a comprehensive understanding of your test coverage. This page explains each metric and how to interpret them.

​
Core Coverage Metrics

​
Total Coverage

Total Coverage measures the percentage of executable lines in your entire codebase that are covered by tests. It’s calculated using the formula: 
Total Coverage = Covered Lines / Executable Lines
Lines that are not executable (comments, blank lines, etc.) are excluded from this calculation. When to use it:
  • As a high-level indicator of overall project health
  • For tracking long-term trends in test coverage
  • For setting minimum coverage thresholds for the entire codebase
Example: A project with 1,500 covered lines out of 2,000 executable lines has a Total Coverage of 75%.

​
Diff Coverage

Diff Coverage focuses only on the lines changed in a pull request or commit. It shows the percentage of new or modified executable lines that are covered by tests. 
Diff Coverage = Covered Changed Lines / Executable Changed Lines
When to use it:
  • For evaluating the test coverage of specific changes
  • As a quality gate for pull requests
  • To ensure new code maintains or improves coverage
Example: A pull request that adds 50 lines of code with 45 of them covered by tests has a Diff Coverage of 90%.

​
Total Coverage Delta

Total Coverage Delta shows how the Total Coverage has changed between two points, typically between the current branch and the base branch in a pull request. 
Total Coverage Delta = New Total Coverage - Previous Total Coverage
When to use it:
  • To understand the impact of changes on overall project coverage
  • To prevent merging changes that significantly decrease coverage
  • To track progress toward coverage goals
Example: If the base branch has 75% coverage and your changes result in 76% coverage, the Total Coverage Delta is +1%.

​
File-level Metrics

​
File Coverage

File Coverage shows the percentage of executable lines in a specific file that are covered by tests. 
File Coverage = Covered Lines in File / Executable Lines in File
When to use it:
  • To identify specific files that need more tests
  • To prioritize test writing efforts
  • To ensure critical files have high coverage
Example: A file with 90 covered lines out of 100 executable lines has a File Coverage of 90%.

​
Directory Coverage

Directory Coverage shows the average coverage across all files in a directory. 
Directory Coverage = Sum of Covered Lines in Directory / Sum of Executable Lines in Directory
When to use it:
  • To understand coverage at the module or component level
  • To identify areas of the codebase that need more testing
  • To set coverage goals for specific components
Example: A directory containing files with a total of 500 covered lines out of 800 executable lines has a Directory Coverage of 62.5%.

​
Coverage Ratings

Qlty assigns letter grades to your coverage percentages to provide an easy-to-understand quality indicator.
Coverage PercentageQlty Coverage Rating
>= 90%A
>= 80%B
>= 70%C
>= 60%D
< 60%F
These ratings appear on your project dashboard and in coverage badges.

​
Advanced Metrics

​
Coverage by Tag

When using Coverage Tags, Qlty tracks separate coverage metrics for each tag, allowing you to see coverage from different test types. Example:
  • Unit test coverage: A (95%)
  • Integration test coverage: B (85%)
  • E2E test coverage: C (75%)

​
Coverage Over Time

Qlty tracks how your coverage metrics change over time, allowing you to see trends:
  • Coverage Trend: Graph showing total coverage over time
  • Weekly Change: How coverage has changed in the last week
  • Monthly Change: How coverage has changed in the last month
  • Quarterly Change: How coverage has changed in the last quarter

​
Coverage Hotspots

Coverage Hotspots identify files with low coverage that are frequently changed, highlighting areas where adding tests would have the most impact. The hotspot score is calculated based on:
  • How frequently the file changes
  • How low the file’s coverage is
  • How important the file is (based on interconnectedness)

​
Interpreting Coverage Metrics

​
What’s a “Good” Coverage Percentage?

The ideal coverage percentage depends on your project’s context:
  • 90%+ (A): Excellent for critical libraries, core business logic, and open-source projects
  • 80-90% (B): Very good for most application code
  • 70-80% (C): Acceptable for many projects
  • 60-70% (D): May indicate testing gaps
  • Below 60% (F): Usually indicates insufficient testing

​
When to Focus on Different Metrics

  • Total Coverage: For overall project health and long-term trends
  • Diff Coverage: For ensuring new code is well-tested
  • File Coverage: For identifying specific areas that need tests
  • Directory Coverage: For understanding module-level testing quality
  • Coverage Deltas: For preventing coverage regression

​
Limitations of Coverage Metrics

While coverage metrics are valuable, they don’t tell the whole story:
  • High coverage doesn’t guarantee high-quality tests
  • Some code may be inherently hard to test (e.g., UI, error handling)
  • Not all code is equally important to test
  • Coverage doesn’t measure test effectiveness at finding bugs
Use coverage as one of several metrics to evaluate your testing strategy, not as the only goal.

​
Best Practices

​
Setting Realistic Goals

  • Start with achievable coverage goals and gradually increase them
  • Set higher coverage requirements for critical code
  • Focus on Diff Coverage for new code rather than trying to immediately increase Total Coverage

​
Effective Coverage Standards

Typical coverage standards might include:
  • All new code must have at least 80% Diff Coverage
  • Critical modules must maintain at least 90% coverage
  • Total Coverage should not decrease

​
Using Coverage Gates

Configure Coverage Gates to enforce your coverage standards:
  • Diff Coverage Gate: Ensures new code meets coverage standards
  • Total Coverage Gate: Prevents significant coverage decreases
  • Custom Gates: For specific requirements like “100% coverage for security code”

​
See Also