Coverage In Your Workflow

Commit Statuses

Qlty can report code coverage metrics as GitHub commit statuses, making it easy to see at a glance if your code meets your coverage requirements. This guide explains how to set up and configure coverage-based commit statuses.

What are Commit Statuses?

Commit statuses are indicators displayed on GitHub pull requests that show whether certain checks or tests have passed or failed. Qlty can add statuses based on your code coverage metrics, helping you enforce quality standards.

Available Coverage Statuses

Qlty provides two types of coverage statuses:

Total Coverage

This status shows the total code coverage percentage for your pull request, and its delta as compared to the base branch.

You can configure it to fail if:

  • Total coverage decreases compared to the base branch

Diff Coverage

This status shows the code coverage percentage only for lines changed in the pull request. You can configure it to fail if:

  • Diff coverage is below a minimum threshold

Enabling Coverage Statuses

To enable coverage statuses:

  1. Go to your project in Qlty
  2. Navigate to “Project Settings” > “Commit Statuses”
  3. Turn on the Total Coverage Gate and/or Diff Coverage Gate
  4. Configure the thresholds according to your requirements

Configuration Options

Total Coverage Options

  • Total Coverage Variation: If total coverage drops by at least this percentage, Qlty will post a failed status to this commit.

Diff Coverage Options

  • Diff coverage: The minimum coverage required for changed lines
  • Minimum Diff Size: The minimum number of lines that must be changed for this status to apply

Example Configurations

Strict Coverage Policy

This configuration enforces high standards for both total and differential coverage:

  • Qlty Coverage:
    • Total Coverage Variation: 1%
  • Qlty Diff Coverage:
    • Minimum diff coverage: 90%
    • Minimum lines: 5

Gradual Improvement Policy

This configuration allows for gradual improvement while still maintaining standards:

  • Qlty Coverage:
    • Total Coverage Variation: 1%
  • Qlty Diff Coverage:
    • Minimum diff coverage: 30%
    • Minimum lines: 5

Branch Protection Rules

To fully enforce your coverage requirements, integrate these statuses with GitHub branch protection rules:

  1. Go to your repository on GitHub
  2. Navigate to “Settings” > “Rules” > “Rulesets”
  3. Edit or create a branch ruleset
  4. Under “Rules” check “Require status checks to pass”, click “Add Checks” and search for and select:
    • “Qlty Coverage”
    • “Qlty Diff Coverage”
  5. Save your changes

Now, pull requests cannot be merged unless they meet your coverage requirements.

Troubleshooting

Status Not Appearing

If you don’t see the commit status:

  • Ensure your Qlty GitHub App has the necessary permissions
  • Verify that a coverage report was successfully uploaded for the commit
  • Check that the coverage gates are enabled in your project settings

Status Always Failing

If your status is always failing:

  • Review your threshold settings and adjust if they’re too strict
  • Check if your tests are running correctly and generating accurate coverage data
  • Verify the coverage report is being uploaded correctly

Status Showing “N/A”

If your status shows “N/A”:

  • For diff coverage, this may happen if the number of changed lines is below your minimum threshold
  • It can also occur if there are no executable lines in the changed files (e.g., only comments or configuration changes)

Best Practices

  • Start with realistic thresholds: Begin with attainable goals and gradually increase them
  • Apply different thresholds for different projects: New projects can aim for higher coverage than legacy codebases
  • Use both total and diff coverage: Enforce high standards for new code while gradually improving overall coverage

See Also