Lab: DevOps telemetry, alerts, indicators, and tracing

Outcome

Correlate a tagged deployment to application telemetry, inspect platform indicators, create GitHub and Azure DevOps failure notifications with an owner, and trace one failed distributed request from request through dependency and exception.

Do

1. Instrument a disposable web workload with Application Insights/OpenTelemetry and send deployment metadata or a release annotation. Run a baseline, deploy a deliberately slower training change, compare performance/failure/usage before and after, then roll back. Save a KQL query with a bounded time window and expected result.
2. In Application Insights, locate one failed request and open transaction diagnostics; follow its operation/correlation ID to dependency and exception telemetry. Fix the training failure and confirm the corrected trace. Inspect VM, Container, Storage, and Network Insights for available training resources; explicitly record a not-applicable family rather than inventing data.
3. Build a dashboard/query showing CPU, memory, disk, and network indicators with time range, aggregation, threshold, and diagnosis. Deliberately select a misleading time range or aggregation, correct it, and record the different conclusion.
4. Create a GitHub Project chart/insight for the training issue flow and interpret its filter. Enable GitHub Actions notification for failed runs to the accountable test user or scoped channel. In Azure DevOps, configure Project settings > Notifications > A build fails for the training pipeline and test a deliberate failed run. If using a service hook, inspect delivery history, retry behavior, and receiver status; remove it after test.

Verify

Capture release/deployment correlation, baseline/change comparison, saved KQL, correlated request/dependency/exception trace, each available Insights view, CPU/memory/disk/network dashboard, GitHub chart, and failed-run notifications from both platforms. You should now understand: deployment correlation links a change to observed behavior; infrastructure indicators need context; tracing follows one operation across components; alert ownership turns a failed run into a response.

Break and fix

1. Remove or mismatch release metadata; diagnose time window, component, deployment identifier, and data collection before restoring correlation.
2. Trigger a dependency failure; follow the trace rather than guessing from CPU, fix the dependency/configuration, and verify the next request.
3. Disable or misroute a pipeline notification; inspect workflow subscription/project notification/service-hook history, correct scope/owner, and retest.

Scenario practice

1. Configure: Choose telemetry source, release correlation, indicator threshold, trace sampling/retention, dashboard/chart, alert route, and incident owner.
2. Diagnose: A release shows high failure rate but normal CPU. Inspect deployment correlation, request/dependency/exception trace, network/disk/memory data, query scope, and rollout cohort.
3. Distinguish: Contrast metrics, logs, Insights, Application Insights, distributed trace, GitHub Project chart, workflow notification, Azure DevOps notification, and service hook.

Clean up

Remove test alerts/notifications/service hooks/charts, delete telemetry/workspace resources only after required evidence is recorded, and remove the disposable workload.

Sources and re-audit

• Application Insights transaction diagnostics, checked 2026-08-22.
• Azure Monitor Insights overview, checked 2026-08-22.
• GitHub Actions workflow notifications, checked 2026-08-22.
• Azure Pipelines failed-run notifications, checked 2026-08-22.

Re-audit when telemetry collection, trace correlation, platform notifications, or dashboard/Insights behavior changes.

Related

application-insights · azure-monitor-insights · github-project-insights · alerting · telemetry · All labs