Azure Automation: validate a scheduled runbook before it changes production

A scheduled Azure Automation runbook easily becomes an operational blind spot. It usually starts with a valid goal: clean a resource, restart a blocked process, correct configuration drift, run a nightly hygiene task or prepare a recurring maintenance action. Once it is connected to a schedule, a managed identity and Azure permissions, it can change production when no operator is actively watching.

The use case is a runbook named aa-prod-maintenance scheduled every night for application resources in rg-app-prod. It must read state, change only objects already marked as eligible, publish usable evidence and stop when the selected scope is larger than expected. The goal is not to write a perfect script. It is to prove that trigger, identity, parameters, logs and rollback are controlled before enabling the production schedule.

Treat the job as a production action

A scheduled runbook is not only a PowerShell or Python file. It is a production action with a trigger, an identity, a scope, expected effects and failure modes. Write that contract before reviewing the code.

Runbook
Name: aa-prod-maintenance
Environment: production
Trigger: schedule nightly-maintenance-prod
Identity: mi-aa-prod-maintenance
Target scope: rg-app-prod only
Allowed action: update resources already tagged maintenance=approved
Forbidden action: create broad role assignment, delete resource, change network exposure

Expected evidence
Input parameters
Target resources selected
Dry-run diff
Real action result
Correlation id
Rollback instruction
Operator handover note

This prevents a common mistake: validating the script while ignoring the execution mode. A runbook launched manually by an administrator does not carry the same risk as a scheduled job running under a persistent identity.

Separate trigger, identity and target

Three controls must stay separate. The schedule decides when the job starts. The identity decides what it can do. Parameters and filters decide what it will touch. When those layers compensate for each other, rollback becomes hard to reason about.

Trigger
Schedule name and timezone
Disabled by default until validation
One production schedule per environment
No public webhook without explicit justification

Identity
Managed identity dedicated to the runbook family
Role assignments scoped to resource group or resource
No inherited broad contributor rights at subscription level
Sign-in and activity logs reviewed during test

Targeting
Explicit subscription and resource group
Required tag or allowlist
Maximum number of resources per run
Stop when the selection is empty or unexpectedly large

The useful test is to break one layer on purpose. If the identity is too broad, the filter selects too much or the schedule points to the wrong environment, the job should fail before the action.

Run a dry-run that leaves evidence

A dry-run should not be a comment in the code. It should produce a readable artifact: resources found, action that would be applied, selection reason, exclusion reason and blocking conditions.

param(
[string]$SubscriptionId,
[string]$ResourceGroupName,
[switch]$DryRun = $true,
[int]$MaxTargets = 10
)

Set-AzContext -SubscriptionId $SubscriptionId | Out-Null

$targets = Get-AzResource -ResourceGroupName $ResourceGroupName |
Where-Object { $_.Tags["maintenance"] -eq "approved" }

if ($targets.Count -eq 0) {
Write-Output "decision=stop reason=no-approved-target"
exit 0
}

if ($targets.Count -gt $MaxTargets) {
throw "Guardrail blocked run: selected $($targets.Count) targets, max is $MaxTargets"
}

$targets | Select-Object Name, ResourceType, ResourceGroupName, Tags |
ConvertTo-Json -Depth 5 |
Write-Output

if ($DryRun) {
Write-Output "decision=preview-only action=none"
exit 0
}

Write-Output "decision=apply targetCount=$($targets.Count)"

Run the dry-run with the same identity as the final job. Otherwise the team validates local administrator behavior, not the scheduled runbook.

Read logs as a decision journal

Azure Automation emits jobs, streams and errors. They should be enough to reconstruct the decision without opening the source code. Add a correlation identifier and short messages for scope, target count, dry-run or apply mode, block reason and result.

let RunbookName = "aa-prod-maintenance";
let WindowStart = datetime(2026-07-03 00:00:00);
let WindowEnd = datetime(2026-07-03 02:00:00);
AzureDiagnostics
| where TimeGenerated between (WindowStart .. WindowEnd)
| where Category has_any ("JobLogs", "JobStreams")
| where RunbookName_s == RunbookName
| project TimeGenerated, RunbookName_s, JobId_g, StreamType_s, ResultDescription
| order by TimeGenerated asc

The exact query depends on Diagnostic Settings and the Log Analytics destination. The evidence expectation is the important part: a reviewer should see whether the job selected the expected resources, whether a guardrail blocked the action or whether the runbook applied a change.

Verify the real identity before widening permissions

Runbook failures are often answered by adding permissions. That is risky when the team has not proved which identity really called Azure Resource Manager. Before widening a role, connect the job to its managed identity and read the activity trail.

AUTOMATION_ACCOUNT="aa-prod-ops"
RESOURCE_GROUP="rg-automation-prod"

az automation account show --name "$AUTOMATION_ACCOUNT" --resource-group "$RESOURCE_GROUP" --query "{name:name,principalId:identity.principalId,type:identity.type}" --output json

az role assignment list --assignee "<managed-identity-principal-id>" --all --query "[].{role:roleDefinitionName,scope:scope}" --output table

The healthy decision is not “make it Contributor”. It is to reduce the role to the useful scope, then replay the dry-run. If the dry-run cannot list the expected resources with the target role, the issue is an authorization contract problem, not something to bypass during an incident.

Keep a simple stop switch

A scheduled job must be easy to neutralize without deleting the runbook. The fastest operational rollback is often to disable the schedule, force DryRun=true, or temporarily remove the write role from the managed identity.

Stop card
Disable schedule nightly-maintenance-prod
Set runbook parameter DryRun=true
Keep the runbook code deployed for investigation
Remove write role from mi-aa-prod-maintenance if action risk remains
Preserve last job logs and input parameters
Open manual procedure for the affected maintenance action

Validation after stop
No new job starts from the schedule
Existing running job is completed, stopped or suspended intentionally
Activity Log shows no further write from the managed identity
Operators know the manual fallback

Avoid rollbacks that destroy evidence. Deleting the runbook or identity may make the incident harder to understand. Suspend the action, keep the traces and reduce permissions.

Decide when to enable the schedule

Enabling the schedule should be a decision, not the final line of deployment. The runbook can be published without being scheduled. The production schedule should only be enabled when contract, dry-run, identity and rollback have been validated together.

Go
The schedule targets the right timezone and environment
Dry-run ran with the production managed identity
Target selection is bounded by tag or allowlist
Maximum target count guardrail is enforced
Logs are visible from the operations workspace
Stop card has been tested

No-go
Identity has broad subscription-level rights
Dry-run was executed from an administrator workstation only
Targeting depends on naming convention alone
Logs are missing or not correlated to the job
Rollback requires deleting the automation account
Operators cannot explain what the next scheduled run will touch

This go/no-go can be attached to the change ticket. It turns an automation that “should work” into a verifiable production operation.

Conclusion

A scheduled Azure Automation runbook should be operated as an autonomous production action. Code matters, but it is not enough. Validation must cover schedule, managed identity, resource scope, dry-run, logs, volume guardrail and stop path.

The decision is straightforward: enable the schedule only when an evidence run shows what will be touched and how to return. Until then, the runbook may stay published, but it should remain in dry-run or disabled mode until the action is observable, bounded and reversible.