Pipeline evaluation and quality gates
MS Learn module: Test & enforce quality in the pipeline · Learning objectives: automate a test suite in CI, measure code coverage, and add build-time quality gates that block low-quality merges before they reach production.
Overview
A pipeline that only compiles is a smoke test, not quality. This module turns CI into a guardrail: it runs the automated test suite, measures coverage of the source, and uses quality gates so a drop in coverage (or a failing test) fails the job and blocks the PR/merge.
Units / lessons covered
- Running unit + integration tests as pipeline steps (
pytest,jest,nunit,jasmine) - Collecting and publishing test results (Azure Pipelines
PublishTestResults; GitHub Actions test adapters) - Code coverage: line vs branch coverage,
publish-code-coverage-results, thresholds - Build quality gates: coverage % floor, defect counts, gate-by-warning policy
- Enforcing gates cross-repo; linking results back into the PR/merge review (see pipeline-run)
Concepts introduced (link to term notes)
Key terms / commands
PublishTestResults·PublishCoverageResults/PublishCodeCoverageResults·coverage.tsv· threshold- GitHub equivalents:
jest-auto-coverage/jacocoadapters ·tests.outcome·metrics.totalCoverage
Hands-on / what to try
Add unit tests to a JS repo, run them in a test job, publish results, and configure a coverage gate of 80% that fails the job if breached. Confirm the PR shows a red checkmark / blocked merge until coverage recovers.
Exam focus
- Quality gates are evaluated per-job; a coverage or test failure aborts the run and blocks downstream stages unless
continue-on-erroris set. - Know the difference between line vs branch (MC/DC) coverage and that coverage alone does not prove behavior — pair it with assertions.
- Both platforms let results flow back into the PR review UI (
checks,statuses).
Related
Path MOC · advanced-yaml-pipelines · Approvals and gates · pipeline-run-diagnostics-and-maintenance
📘 Source: Publish test results · Publish code coverage results