Azure Automation: prove the runbook identity before broadening RBAC

An Azure Automation runbook that worked yesterday suddenly returns 403 Forbidden. The job reaches Azure, managed identity is enabled and a role assignment exists on the target. Granting Contributor to the Automation account looks like a quick recovery. It can also hide an identity mismatch and give the wrong principal durable production access.

The use case is a PowerShell runbook that may execute in an Azure sandbox or on a Hybrid Runbook Worker. A managed identity was added to the Automation account or worker VM, then the job lost access to a vault, VM or target resource. This runbook drives a bounded decision: correct principal selection or scope, preserve the denial, promote through a canary, or restore the previous identity path.

Freeze one job and one denied action

Do not start from the IAM blade. Preserve an exact job, published version, execution target, UTC window and denied operation. An ARM 403, a Key Vault data-plane denial and a local Hybrid Worker failure require different evidence and permissions.

incident:
automation_account: aa-ops-prod
runbook: rotate-app-certificate
job_id: <job-guid>
started_at_utc: 2026-08-30T14:20:00Z
execution_target: cloud-or-hybrid-worker-group
hybrid_worker: <worker-name-or-not-applicable>
published_runbook_version: <commit-or-version>

denied_action:
resource_id: <exact-resource-id>
operation: <provider/action-or-data-action>
http_status: 403
correlation_id: <arm-or-service-correlation-id>

expected_identity:
type: automation-account-system-assigned-or-user-assigned-or-worker-vm
client_id: <expected-client-id>
principal_id: <expected-object-id>

stop_conditions:
- execution target is unknown
- denied operation is not identified
- proposed role scope is subscription-wide
- rollback identity is already disabled

Preserve job streams before retrying. A retry may land on a different worker, load a different context or emit a new correlation ID. The first objective is not success. It is an attributable failure.

Map the intended identity path

In a cloud sandbox, Connect-AzAccount -Identity normally uses the Automation account managed identity. A user-assigned identity can be selected explicitly by client ID. Identity selection on an Azure Hybrid Runbook Worker is more subtle: the Automation account identity can take precedence over the VM identity, and an account user-assigned identity is not available through the same worker path. The VM can provide its own managed identity only when account configuration and runbook code leave that path effective.

Write the expected chain before inspecting role assignments.

Cloud sandbox
Connect-AzAccount -Identity
-> system-assigned identity of the Automation account

Cloud sandbox with explicit user-assigned identity
Connect-AzAccount -Identity -AccountId <client-id>
-> selected user-assigned identity attached to the Automation account

Hybrid Worker on an Azure VM
Automation account identity enabled
-> verify that the account system-assigned identity is the effective principal

Hybrid Worker using VM identity
-> prove the account configuration permits this path
-> select the VM system-assigned or user-assigned identity explicitly

Never infer identity from
runbook name, worker name, resource naming or an existing role assignment

A role on mi-runbooks-prod proves nothing until the runtime is shown to use that principal. Granting the same role to every candidate identity only turns an authentication incident into persistent security debt.

Reset Az context inside the process

A reused worker can retain an Az context from an earlier execution. A runbook that skips explicit authentication may then operate under an unexpected context. Disable process-level context autosave, authenticate the chosen identity and pass the resulting context to sensitive cmdlets.

$ErrorActionPreference = 'Stop'
Disable-AzContextAutosave -Scope Process
Clear-AzContext -Scope Process -Force -ErrorAction SilentlyContinue

$expectedSubscriptionId = '<subscription-id>'
$expectedTenantId = '<tenant-id>'
$userAssignedClientId = '<empty-or-user-assigned-client-id>'

if ([string]::IsNullOrWhiteSpace($userAssignedClientId) -or
  $userAssignedClientId -eq '<empty-or-user-assigned-client-id>') {
$context = (Connect-AzAccount -Identity -Tenant $expectedTenantId).Context
} else {
$context = (Connect-AzAccount -Identity -AccountId $userAssignedClientId -Tenant $expectedTenantId).Context
}

$context = Set-AzContext -SubscriptionId $expectedSubscriptionId -Tenant $expectedTenantId -DefaultProfile $context

[pscustomobject]@{
AccountId      = $context.Account.Id
AccountType    = $context.Account.Type
SubscriptionId = $context.Subscription.Id
TenantId       = $context.Tenant.Id
Environment    = $context.Environment.Name
} | ConvertTo-Json -Compress

if ($context.Subscription.Id -ne $expectedSubscriptionId) {
throw 'Unexpected subscription context. Stop before any write.'
}

Do not log tokens, secrets or complete claims. Account, tenant, subscription, job and correlation identifiers establish the operational chain without exposing credentials.

Match runtime evidence to configured principals

Inventory the system-assigned principalId, the clientId and principalId of user-assigned identities, and identities attached to the worker. Do not confuse a client ID, service principal object ID and managed identity resource ID.

SUBSCRIPTION_ID="<subscription-id>"
AUTOMATION_RG="rg-automation-prod"
AUTOMATION_ACCOUNT="aa-ops-prod"
TARGET_RESOURCE_ID="<exact-target-resource-id>"

az account set --subscription "$SUBSCRIPTION_ID"

az automation account show --resource-group "$AUTOMATION_RG" --name "$AUTOMATION_ACCOUNT" --query '{id:id,systemPrincipalId:identity.principalId,userAssigned:identity.userAssignedIdentities}' --output json

EXPECTED_PRINCIPAL_ID="<object-id-proved-by-runtime-path>"

az role assignment list --assignee-object-id "$EXPECTED_PRINCIPAL_ID" --scope "$TARGET_RESOURCE_ID" --include-inherited --query '[].{role:roleDefinitionName,scope:scope,principalType:principalType}' --output table

An empty list at the exact resource does not rule out inherited assignments or Entra group membership. Conversely, a visible assignment may not contain the denied action or data action. Compare the exact operation with the role definition.

Separate control plane from data plane

The job may read ARM configuration while being denied by the resource data plane. Reader on a vault does not grant secret access. A Storage data role does not permit firewall changes. Classify the refusal before changing access:

• Azure Resource Manager operation against resource configuration;
• data-plane operation against a secret, blob, queue or database;
• Microsoft Graph or Entra operation with a separate authorization model;
• local or network access from the Hybrid Worker.

This prevents using a broad ARM role to repair a data-plane denial, or changing RBAC when the worker simply cannot resolve or reach the endpoint.

Correlate the denial with execution evidence

Align the UTC window, job ID, target, operation and correlation ID. For ARM operations, Activity Log evidence can confirm the caller and reached scope. For data-plane operations, use the service diagnostics and runbook traces. A lone 403 in the error stream is not enough.

let StartTime = datetime(2026-08-30T14:15:00Z);
let EndTime = datetime(2026-08-30T14:35:00Z);
let TargetResourceId = tolower("<exact-target-resource-id>");
AzureActivity
| where TimeGenerated between (StartTime .. EndTime)
| where tolower(_ResourceId) == TargetResourceId
| where ActivityStatusValue =~ "Failure" or tostring(Properties) has "403"
| project TimeGenerated, OperationNameValue, ActivityStatusValue,
        Caller, CallerIpAddress, CorrelationId, ResourceGroup, Properties
| order by TimeGenerated asc

If the observed caller does not match the expected principal, stop the RBAC change. Correct identity selection or execution targeting first. If it matches, continue with the exact denied action, role definition, scope and propagation delay.

Build the smallest testable correction

The correction must be a reviewable diff, not an unbounded temporary elevation. Prefer, in order: explicitly select the intended identity, remove reliance on persisted context, correct a scope that is demonstrably too narrow, then choose a more precise role containing the required action.

candidate:
runtime: cloud-sandbox-or-hybrid-worker
expected_principal_id: <object-id>
denied_operation: <provider/action-or-data-action>
target_scope: <smallest-resource-or-resource-group-scope>
proposed_role: <least-privilege-role>
previous_identity_path: <documented-path>

canary:
runbook: identity-readonly-canary
input_target: <non-critical-resource>
allowed_action: <read-or-dry-run-operation>
forbidden_action: <write-operation-that-must-still-fail>

promote_when:
- runtime principal matches the expected object ID
- allowed action succeeds at the intended scope
- forbidden action remains denied
- production runbook uses an explicit Az context
- job and service logs share a correlation window

rollback_when:
- a different principal appears
- access extends to an unintended scope
- hybrid and cloud jobs resolve different identity paths
- the denied action cannot be explained

The canary must prove one allowed action and one action that remains denied. Success alone does not demonstrate that the permission boundary still holds.

Promote and watch for side effects

Apply the change through the IaC or workflow that owns the assignment. Allow for propagation, then run the canary on the same execution target as production. Follow with one bounded production action against one resource.

Watch for new 403 responses, but also successful access outside the target scope, execution on a different worker, subscription context changes and operations that were previously impossible. Fewer errors are not proof if the principal now has excessive access.

Decide correction, preserved denial or rollback

Correct identity selection when the observed caller violates the contract. Correct role or scope only after proving the principal and showing that a required operation is genuinely missing. Preserve the denial when the operation is outside the runbook mandate or the target remains ambiguous.

Roll back when identity differs between sandbox and Hybrid Worker, the canary exposes a wider scope, or actions can no longer be attributed. Rollback restores the versioned identity path and assignments. It does not stack another emergency role on top.

Conclusion

An Azure Automation 403 is not automatically a request for more privilege. It first asks an attribution question: which environment executed the job, which principal acquired the token, which operation was denied and at what scope.

Freeze the job, reset Az context, prove the runtime principal, separate control plane from data plane, then test the smallest correction with positive and negative canaries. The desired outcome is more than a green job: it is an explainable, observable and reversible identity chain.