Lab: Flow, quality, security, delivery, and operations metrics

Outcome

Build a decision-oriented dashboard from known work, PR/build/test/security/deployment/incident data; document each metric’s source, filter, owner, decision threshold, and anti-gaming limit.

Do

1. Create a shared Azure Boards query and Analytics view for a fixed iteration/area. Add dashboard charts for backlog age/throughput, cycle time, lead time, and work in progress. Remember that Analytics views cover Azure Boards work items, not pipeline data.
2. Add or link separate evidence for PR review time/build success, test pass/duration/flaky tests, security finding age/severity/exception owner, deployment frequency/change failure, and alert-to-resolution/MTTR timeline. Use pipeline, test, security, deployment, and Monitor/incident sources rather than pretending a Boards view contains them.
3. For every chart, write: question, numerator/denominator or event boundaries, time window/filter, owner, threshold, and next action. Deliberately select a misleading filter (for example, exclude reopened work or failed deployments), show its false conclusion, correct it, and record the decision change.

Verify

Capture dashboard/query/view settings and the data source, calculation, filter, interpretation, and decision for planning, development, testing, security, delivery, and operations. You should now understand: a dashboard is a decision surface; a metric without scope, event boundaries, and action can mislead or be gamed.

Break and fix

1. Filter out reopened items or failed deployments; identify the biased sample and correct it.
2. Use an Analytics view to seek pipeline data; diagnose the source limitation and add the appropriate pipeline/test/deployment source.
3. Set an incentive based only on ticket count; add quality, outcome, or escape-rate context before acting.

Scenario practice

1. Configure: Define cycle/lead/MTTR boundaries, data sources, thresholds, owner, and review cadence.
2. Diagnose: Deployment frequency improved while change failure rose. Validate source data, cohort/filter, rollback classification, and incident linkage.
3. Distinguish: Contrast planning, development, test, security, delivery, and operations metrics; contrast a dashboard widget, query, Analytics view, and Power BI report.

Clean up

Remove test views/widgets/queries and redact training identifiers from any shared dashboard.

Sources and re-audit

• Dashboards, charts, reports, and widgets, checked 2026-08-22.
• Analytics views, checked 2026-08-22.

Re-audit when Analytics scope, dashboard sources, or metric instrumentation changes.

Related

devops-flow-metrics · github-project-insights · test-coverage · github-advanced-security · All labs