Automation
Azure DevOps: validate approvals and checks before bypassing production
A production runbook for qualifying an Azure DevOps deployment blocked by approvals or checks with environment scope, identity, logs, audit evidence, validation and rollback before bypassing the guardrail.
An Azure DevOps deployment blocked on an approval, an environment check or an external validation often creates immediate pressure: approve manually, disable the check, rerun the pipeline, change the service connection identity or push the fix through a parallel path. Those actions may unblock the release, but they can also remove the only barrier that prevented an unqualified change from reaching production.
The use case is an Azure DevOps pipeline deploying an application or Azure infrastructure to a production environment. The run is waiting on an approval, a business-hours check, a REST validation, branch control, a service connection or a policy that no longer passes. The runbook goal is to decide whether the block is a legitimate protection, a guardrail incident, identity drift or a signal problem before bypassing anything.
Name the guardrail that blocks the run
Start by isolating the guard type and its scope. A “blocked pipeline” is not an actionable diagnosis. The team needs to know which decision is expected, who owns it, what evidence is required and what risk is introduced if the guardrail is bypassed.
Deployment to qualify
Azure DevOps project: platform-prod
Pipeline: deploy-orders-api
Run: 20260710.4
Blocked stage: production
Environment: prod-orders
Change: application image + App Configuration settings
Active guardrail: manual approval + REST check + branch control
Deployment identity: service connection sc-prod-orders
Window: planned production fix
Evidence required before bypass
Commit, artifact and change plan identified
Exact reason for pending, failed or expired check
Expected approver and service owner confirmed
Real service connection identity verified
Pipeline and external check logs retained
Application validation and rollback documented
Written decision: approve, rerun, fix the check, roll back or blockIf the team cannot explain the guardrail, it should not remove it. An approval gate is part of the production system, not an administrative nuisance.
Separate normal waiting, failure and configuration drift
Three situations look similar during an incident: the pipeline is waiting for a normal approval, the check failed because it detected a real risk, or the check mechanism itself is broken. They require different responses.
Normal waiting
The run is waiting for a known approver
The change window is not open yet
The external check has not returned a verdict yet
No timeout or technical error is visible
Legitimate guardrail failure
Branch, artifact or environment does not satisfy the rule
The check detects a failed test, vulnerability or missing evidence
The service connection does not match the expected scope
The approver rejects or asks for additional proof
Guardrail drift or outage
The approver no longer exists or no longer has access
The REST check expires without reaching its service
A policy points to an obsolete branch or group
The service connection was changed outside a planned change
Check logs are no longer exported or correlatableBypass is only a candidate in the third case, and only with a documented compensating control. In the first two cases, the correct action is to wait or fix the reason for rejection.
Capture the run and artifact before any rerun
Before rerunning or approving, freeze what will actually go to production. A new run can rebuild a different artifact, pick up a changed variable or use a modified service connection.
ORG="https://dev.azure.com/example"
PROJECT="platform-prod"
RUN_ID="20260710"
PIPELINE_ID="42"
az pipelines runs show --org "$ORG" --project "$PROJECT" --id "$RUN_ID" --query "{id:id,name:name,state:state,result:result,sourceBranch:sourceBranch,sourceVersion:sourceVersion,createdDate:createdDate,finishedDate:finishedDate}" --output json
az pipelines runs artifact list --org "$ORG" --project "$PROJECT" --run-id "$RUN_ID" --output table
az pipelines show --org "$ORG" --project "$PROJECT" --id "$PIPELINE_ID" --query "{id:id,name:name,folder:folder,revision:revision}" --output jsonKeep the commit SHA, artifact, stage and environment name in the incident notes. Approval on “the latest run” is not precise enough for production.
Read the environment checks as operational code
Approvals and checks attached to an Azure DevOps environment should be treated as critical configuration. Before modifying them, review the rule: approver group, timeout, deferred approval, branch checks, external requests, exclusive lock or business hours.
environment: prod-orders
checks_to_review:
manual_approval:
approvers: platform-release-managers
timeout: 4h
allow_self_approval: false
required_evidence:
- incident_or_change_id
- artifact_version
- rollback_owner
branch_control:
allowed_branches:
- refs/heads/main
- refs/heads/hotfix/*
verify_protection: true
external_rest_check:
service: change-risk-api
expected_decision: approved | rejected | retryable
timeout: 10m
evidence_required:
- risk_score
- blocking_reason
- correlation_id
exclusive_lock:
scope: environment
reason: prevent_parallel_production_deploymentsIf the rule is too vague to review, it is too vague to bypass. The fix may be a policy update, but only after saving the initial state and the reason for change.
Verify the identity that can really deploy
An approval does not protect much if the service connection or deployment identity has drifted. Check the identity used by the stage, its Azure scope, its permissions and recent changes before approving.
ORG="https://dev.azure.com/example"
PROJECT="platform-prod"
SERVICE_ENDPOINT_ID="00000000-0000-0000-0000-000000000000"
az devops service-endpoint show --org "$ORG" --project "$PROJECT" --id "$SERVICE_ENDPOINT_ID" --query "{name:name,type:type,authorization:authorization.scheme,createdBy:createdBy.displayName,modifiedBy:modifiedBy.displayName,isShared:isShared,serviceEndpointProjectReferences:serviceEndpointProjectReferences[].projectReference.name}" --output json
az role assignment list --assignee "<principal-id-used-by-service-connection>" --scope "/subscriptions/<subscription-id>/resourceGroups/rg-prod-orders" --query "[].{role:roleDefinitionName,scope:scope}" --output tableThe fix should not be to broaden the identity just to pass the deployment. If the identity is too weak, prove the missing role and limit it to the required scope. If it is already too broad, bypass becomes even riskier.
Correlate the external check with application logs
When a REST check or external policy blocks the run, retrieve its exact verdict. A network timeout, a 500 error and a business rejection do not lead to the same decision.
let StartTime = datetime(2026-07-10T08:00:00Z);
let EndTime = datetime(2026-07-10T09:00:00Z);
let CorrelationId = "ado-prod-20260710-04";
AppRequests
| where TimeGenerated between (StartTime .. EndTime)
| where Name has "change-risk-api" or Url has "/deployment/check"
| where tostring(Properties["correlationId"]) == CorrelationId
| project TimeGenerated, Name, Url, ResultCode, DurationMs, Success, OperationId, Properties
| order by TimeGenerated ascIf the external check is unavailable, the compensating control must be explicit: who validates risk, with which data, for how long, and how the policy will be restored. Otherwise the check outage becomes a permanent precedent.
Decide approval, rerun, fix or controlled bypass
The decision must be more precise than “unblock the pipeline”. Keep a short matrix that ties each action to available evidence.
Approve the deployment
Artifact, commit and environment are identified
Checks pass or their waiting state is normal
Approver is authorized and different from the change author
Application or infrastructure rollback is ready
Post-deployment validation is defined
Rerun the pipeline
The run expired without code or configuration change
The artifact can be reused or rebuilt deterministically
The external check is available again
The rerun will not replay a partial action that was not rolled back
Fix the guardrail
Approver, group, branch or check service is obsolete
The policy fix is reviewed as a production change
The initial state is saved
The corrected guardrail is tested on a non-destructive run
Controlled bypass
The check mechanism is unavailable, not the business risk model
The service owner accepts the compensating control
Bypass is limited to the run, stage and window
Audit evidence and a restoration ticket exist
Block
The check rejects for a business or security reason
Artifact or commit is not identified
Deployment identity drifted without review
Rollback is not testable
The requested bypass disables the guardrail for future runsMost gate incidents are resolved by correcting configuration or rerunning in a controlled way, not by globally removing approval.
Validate after promotion and restore the guardrail
If the deployment is approved or bypassed, post-deployment validation must prove two things: the service works and the guardrail has returned to the expected state.
Post-deployment validation
Production stage completed on the expected run
Deployed artifact matches the approved commit
Application probes pass with a correlation ID
Azure Monitor alerts remain within the expected envelope
Azure DevOps audit logs retain approval, rejection, bypass or modification
Environment checks are re-enabled or corrected
Ticket contains decision, evidence, approver, rollback and owner
Rollback or restoration
Application rollback if probes or metrics regress
Policy restoration if a temporary bypass was created
Removal of temporary service connection permissions
Non-destructive run to prove the guardrail works again
Post-incident review if the check stayed unavailable too longA bypass without verified restoration is not a temporary workaround. It is a lasting reduction in control.
Conclusion
An Azure DevOps approval or check that blocks production is not automatically an incident to bypass. It is a signal to qualify: exact run, artifact, environment, rule, approver, identity, external verdict, logs and rollback.
The right exit is a traceable decision: approve with evidence, rerun without changing the artifact, fix the guardrail, bypass one run with compensating control, or block the change. This keeps CI/CD guardrails from becoming switches that are turned off whenever they become inconvenient.