Reference

Troubleshooting

This guide helps you troubleshoot common issues with code coverage in Qlty.

Upload Issues

Coverage Report Not Being Detected

Symptoms:

  • “Unsupported format” errors
  • “Unable to parse coverage report” messages
  • No coverage data appears in Qlty after upload

Potential Causes:

  • Unsupported coverage format
  • Corrupt or incomplete coverage report
  • Incorrect file path in upload command

Solutions:

  1. Verify your coverage format is supported
  2. Check that the coverage report file exists and is complete
  3. Try using the --format flag to explicitly specify the format: 
    $qlty coverage publish --format=lcov path/to/coverage.info
  4. Examine the report file for valid content
  5. Generate a fresh coverage report and try again

Authentication Failures

Symptoms:

  • “Authentication failed” errors
  • “Invalid token” messages
  • “Permission denied” errors

Potential Causes:

  • Incorrect coverage token
  • Expired or rotated token
  • Missing environment variable
  • Using a token with incorrect scope

Solutions:

  1. Verify your token is correct in your CI settings
  2. Check environment variable names (should be QLTY_COVERAGE_TOKEN)
  3. Regenerate your token in Qlty and update your CI configuration
  4. Ensure you’re using the token for the correct workspace or project
  5. If using OIDC, verify the permissions configuration in your GitHub workflow

Path Mapping Issues

Symptoms:

  • Coverage appears but doesn’t match files in your repository
  • “File not found” errors when viewing coverage
  • Coverage shown for files that don’t exist in your repo

Potential Causes:

  • Coverage report contains absolute paths
  • Path structure differs between build environment and repository
  • Generated files with different paths

Solutions:

  1. Use path fixing options: 
    $qlty coverage publish --strip-prefix=/absolute/path coverage.xml
  2. Check that your coverage tool is configured to use relative paths
  3. Inspect the coverage report to see what paths are being reported
  4. Use qlty coverage transform to convert paths before uploading

Missing Metadata

Symptoms:

  • Coverage is uploaded but not associated with the correct branch
  • Coverage doesn’t appear in pull requests
  • Coverage data shows with incorrect commit information

Potential Causes:

  • CI system metadata not detected
  • Shallow git clone in CI
  • Custom CI setup not recognized by Qlty

Solutions:

  1. Manually specify metadata with override flags: 
    $qlty coverage publish --override-branch=main --override-commit-sha=abcdef123 coverage.xml
  2. Ensure your CI environment provides standard environment variables
  3. Use a deeper git clone in CI (or fetch git metadata)
  4. Check that the git repository is available in the CI environment

Viewing Issues

Coverage Not Showing in GitHub

Symptoms:

  • No coverage status checks on pull requests
  • Missing coverage comments on pull requests
  • Coverage data visible in Qlty but not in GitHub

Potential Causes:

  • GitHub app permissions issues
  • Coverage gates not enabled
  • PR comments not configured
  • Timing issues with CI and status reporting

Solutions:

  1. Verify GitHub integration settings in project settings
  2. Check that coverage gates are enabled in quality gates settings
  3. Ensure PR comments are enabled
  4. Check GitHub app permissions in your organization settings
  5. Try manually uploading coverage for the PR branch

Inconsistent Coverage Reports

Symptoms:

  • Different coverage percentages in different views
  • Coverage fluctuates between builds without code changes
  • Coverage shown in CI logs differs from what appears in Qlty

Potential Causes:

  • Different coverage criteria (line vs branch vs statement)
  • Test flakiness or non-deterministic tests
  • Cache issues in tests
  • Different coverage tool configurations

Solutions:

  1. Standardize coverage configuration across environments
  2. Fix flaky tests that produce inconsistent results
  3. Use the same coverage tool and settings consistently
  4. Implement deterministic test ordering
  5. Clear caches between test runs

Missing Coverage for Some Files

Symptoms:

  • Some files show no coverage data
  • Certain directories are missing from coverage reports
  • Coverage appears for some languages but not others

Potential Causes:

  • Include/exclude patterns filtering out files
  • Multi-language projects with incomplete coverage setup
  • Files not being exercised by tests
  • Generated code or third-party code inclusion/exclusion

Solutions:

  1. Check coverage tool configuration for include/exclude patterns
  2. Verify that all relevant files are included in coverage report generation
  3. Set up coverage for each language in multi-language projects
  4. Use multiple coverage reports for different parts of the codebase
  5. Configure path exclusions appropriately in Qlty

Quality Gate Issues

Failing Coverage Gates

Symptoms:

  • Pull requests failing due to coverage checks
  • “Coverage decreased” warnings even with improved test coverage
  • Diff coverage failing despite high coverage in changed files

Potential Causes:

  • Thresholds set too high for your project’s current state
  • Coverage decreasing in files not directly modified
  • Base branch coverage calculation issues
  • Coverage calculation includes excluded files

Solutions:

  1. Adjust coverage gate thresholds to realistic levels
  2. Investigate unexpected coverage changes
  3. Temporarily disable gates while building up coverage
  4. Use path exclusions to ignore files that shouldn’t affect gates
  5. Focus on diff coverage rather than total coverage for incremental improvement

Coverage Badges Not Updating

Symptoms:

  • Outdated coverage percentage in badges
  • Coverage badge shows “unknown” or “N/A”
  • Badge doesn’t reflect recent coverage improvements

Potential Causes:

  • Badge caching
  • No coverage data for default branch
  • Badge URL issues

Solutions:

  1. Ensure coverage is being uploaded for your default branch
  2. Check badge URLs and project settings
  3. Manually refresh cached badges
  4. Verify that the badge is pointing to the correct project and metric
  5. Wait for cache expiration (typically a few hours)

Advanced Issues

Merging Coverage Reports

Symptoms:

  • Coverage from parallel tests not combining correctly
  • Missing coverage data from certain test suites
  • Merged coverage showing lower than expected percentages

Potential Causes:

  • Incorrect merging configuration
  • Incompatible coverage report formats
  • Issues with multi-part uploads

Solutions:

  1. Review coverage merging documentation
  2. For client-side merging, ensure all reports use the same format
  3. For server-side merging, verify the --total-parts-count configuration
  4. Check that all parts are successfully uploaded
  5. Try converting all reports to a common format before merging

Coverage Tag Issues

Symptoms:

  • Tagged coverage not appearing separately
  • Incorrect coverage shown for specific tags
  • Tags not filtering correctly in UI

Potential Causes:

  • Incorrect tag specification during upload
  • Mixing tagged and untagged uploads
  • UI filtering confusion

Solutions:

  1. Verify tag naming in upload commands: 
    $qlty coverage publish --tag=unit coverage.xml
  2. Ensure consistent tag usage across uploads
  3. Check tag configuration in project settings
  4. Use different tags for different test types (unit, integration, etc.)

Large Repository Issues

Symptoms:

  • Coverage uploads timing out
  • Very slow coverage rendering in UI
  • Missing coverage for some files in large repositories

Potential Causes:

  • Coverage reports too large
  • Too many files included in coverage
  • Performance limits in browser extensions

Solutions:

  1. Use path exclusions to limit coverage to relevant code
  2. Split large repositories into smaller projects
  3. Exclude generated code, vendor code, or test code from coverage
  4. Use more specific coverage focus for very large repositories
  5. Contact Qlty support for help with large-scale deployments

Getting Additional Help

If you’ve tried the solutions above and are still experiencing issues:

  1. Check Logs: Use QLTY_DEBUG=true with your commands to get detailed logs
  2. Contact Support: Reach out to Qlty support
  3. Join Discord: Ask for help in our Discord community
  4. GitHub Issues: Submit a bug report on our GitHub issue tracker

When asking for help, include:

  • The exact command you’re running
  • Any error messages (with sensitive information redacted)
  • Your CI provider and configuration
  • The coverage tool and language you’re using
  • Steps you’ve already taken to troubleshoot