Azure Managed Identity: diagnose principal drift after resource recreation

An Azure application is redeployed under the same name, starts normally, then receives 403 responses from Key Vault, Storage or an internal API. The code, hostname and settings look unchanged. The immediate proposal is to restore Contributor at resource-group scope. Yet the resource may have kept its name while changing its security identity.

The running case is an App Service named app-orders-prod, recreated by infrastructure as code during a replacement. Its system-assigned managed identity was deleted with the old resource, and a new service principal was created. Existing role assignments still target the old principalId. This runbook drives a bounded decision: restore minimum roles to the new principal, return to the previous resource if it still exists, or move to a user-assigned identity when authorization must outlive compute.

Freeze the identity contract

Identify the consumer, target and expected permission first. A 403 does not say whether the token is missing, issued for the wrong principal, accepted but underprivileged, or rejected by a network condition.

incident: inc-20260912-004
consumer:
resource: app-orders-prod
resource_id: /subscriptions/<sub>/resourceGroups/rg-orders-prod/providers/Microsoft.Web/sites/app-orders-prod
identity_type: SystemAssigned
deployment: orders-infra-2026.09.12.2
target:
resource: kv-orders-prod
operation: secrets/get
expected_role: Key Vault Secrets User
expected_scope: /subscriptions/<sub>/resourceGroups/rg-security-prod/providers/Microsoft.KeyVault/vaults/kv-orders-prod
symptom:
first_seen_utc: 2026-09-12T05:42:00Z
status: 403
correlation_id: <request-or-correlation-id>
preserve:
- previous_resource_identity_principal_id
- current_resource_identity_principal_id
- role_assignments_for_both_principals
- deployment_and_activity_timeline
- target_data_plane_logs
- last_known_good_iac_revision

Do not change roles yet. The first fact to prove is a principal change, not a generic lack of permission.

Compare the resource identity, not its name

A system-assigned identity shares its lifecycle with its resource. Deleting and recreating that resource produces a new identity even when Azure reuses the resource name and textual resource ID. Roles bind to the principalId, not to the App Service name.

set -eu

RG="rg-orders-prod"
APP="app-orders-prod"
RESOURCE_ID=$(az webapp show -g "$RG" -n "$APP" --query id -o tsv)

az webapp identity show --resource-group "$RG" --name "$APP" --query '{type:type,principalId:principalId,tenantId:tenantId,userAssignedIdentities:userAssignedIdentities}' --output json

az deployment group list --resource-group "$RG" --query '[0:10].{name:name,state:properties.provisioningState,time:properties.timestamp}' --output table

az monitor activity-log list --resource-id "$RESOURCE_ID" --start-time 2026-09-12T04:30:00Z --query '[].{time:eventTimestamp,operation:operationName.value,status:status.value,caller:caller,correlationId:correlationId}' --output table

Recover the former principalId from the previous deployment output, IaC state, identity inventory or change record. Do not select an Entra object only because its display name matches the resource name: deleted and active objects can make that shortcut misleading.

Prove drift in role assignments

Compare direct assignments for both identifiers. Using --assignee-object-id with principal-name lookup disabled lets the check rely on the known identifier without requiring Entra to resolve the object.

set -eu

OLD_PRINCIPAL_ID="<old-principal-id>"
NEW_PRINCIPAL_ID="<new-principal-id>"

for PRINCIPAL_ID in "$OLD_PRINCIPAL_ID" "$NEW_PRINCIPAL_ID"; do
echo "principal=$PRINCIPAL_ID"
az role assignment list   --all   --assignee-object-id "$PRINCIPAL_ID"   --fill-principal-name false   --query '[].{role:roleDefinitionName,roleId:roleDefinitionId,scope:scope,assignmentId:id}'   --output table
done

The diagnosis is strong when the old principal has the expected role at the expected scope, the new principal does not, and recreation immediately precedes the denials. An assignment displayed as Identity not found is not transferable authorization; it still references the former identifier.

Separate token, authorization and network

Do not let a changed principalId become a universal explanation. Confirm that the workload obtains a token and that the target sees the new principal. For a Key Vault using Azure RBAC, data-plane logs can connect caller identity, operation and result. For a vault using access policies, repair belongs in the vault policy rather than an Azure role assignment.

let StartTime = datetime(2026-09-12T05:30:00Z);
let EndTime = datetime(2026-09-12T06:30:00Z);
let VaultName = "kv-orders-prod";
AzureDiagnostics
| where TimeGenerated between (StartTime .. EndTime)
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where Resource =~ VaultName
| where ResultSignature in ("403", "Forbidden") or ResultType =~ "Forbidden"
| extend CallerObjectId = tostring(column_ifexists("identity_claim_oid_g", ""))
| project TimeGenerated, OperationName, ResultSignature, CallerObjectId,
        CallerIPAddress, CorrelationId
| order by TimeGenerated asc

Column names depend on the collection mode and destination table. Inspect one raw event before adapting the query. If no call reaches the vault, return to token acquisition, DNS, firewall and the private path. Creating a role cannot repair a request that never reaches the target.

Restore only the proven contract

Do not clone every role from the old principal. Some assignments may be historical, too broad or tied to a function the replacement no longer performs. Recreate only the validated tuple: new principal, role definition and exact scope.

set -eu

NEW_PRINCIPAL_ID="<new-principal-id>"
TARGET_SCOPE="/subscriptions/<sub>/resourceGroups/rg-security-prod/providers/Microsoft.KeyVault/vaults/kv-orders-prod"
ROLE="Key Vault Secrets User"

az role assignment create --assignee-object-id "$NEW_PRINCIPAL_ID" --assignee-principal-type ServicePrincipal --role "$ROLE" --scope "$TARGET_SCOPE"

az role assignment list --assignee-object-id "$NEW_PRINCIPAL_ID" --scope "$TARGET_SCOPE" --include-inherited --fill-principal-name false --query '[].{role:roleDefinitionName,scope:scope,principalId:principalId}' --output table

In IaC, derive the assignment from the principalId produced by the resource or identity, and use a deterministic role-assignment name that includes scope, role and principal. Avoid a fixed identifier copied from an older deployment; it turns every replacement into a delayed failure.

param targetScopeId string
param roleDefinitionId string

resource app 'Microsoft.Web/sites@2023-12-01' existing = {
name: 'app-orders-prod'
}

resource target 'Microsoft.KeyVault/vaults@2023-07-01' existing = {
name: 'kv-orders-prod'
scope: resourceGroup('rg-security-prod')
}

resource access 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(targetScopeId, roleDefinitionId, app.identity.principalId)
scope: target
properties: {
  principalId: app.identity.principalId
  principalType: 'ServicePrincipal'
  roleDefinitionId: roleDefinitionId
}
}

Adapt API versions and scopes to the repository that owns the infrastructure. The important part is the explicit dependency, not the fragment itself.

Choose system-assigned or user-assigned lifecycle

Rollback does not mean arbitrarily reattaching the old object. A system-assigned identity deleted with its resource is not an independent component to restore. The right action depends on the intended lifecycle.

Keep the system-assigned identity
Resource and permissions should disappear together
Replacements are rare and IaC automatically recreates roles
The current principal is injected without a fixed value

Move to a user-assigned identity
The resource is recreated or swapped regularly
Permissions must survive compute replacement
Several slots or resources deliberately share one contract
Identity attachment remains separate from role administration

Roll back the deployment
Recreation was not intended
Other unmigrated state was lost
The previous resource still exists and its return is tested

Block role restoration
The former principal or expected scope cannot be proven
The replacement workload asks for more rights than before
No target log connects the 403 to the new principal

A user-assigned identity stabilizes the principal but also extends the lifetime of its permissions. It needs an owner, an inventory of attached resources and explicit removal when the contract ends.

Validate, then clean up orphaned assignments

Allow for RBAC propagation, obtain a fresh token and replay one bounded read. Validation must cover the real target and a negative action: the new principal can read the intended secret but cannot write it or read another vault.

Validate
Workload obtains a new token after correction
Target logs show the new principalId
Expected operation succeeds at the exact scope
An out-of-contract operation remains denied
Application probes return healthy
The next IaC plan proposes no broader authorization

Clean up after validation
Inventory role assignments targeting the former principalId
Prove no active token or workload still uses it
Delete orphaned assignments by exact assignment ID
Record old principal, new principal, cause and evidence

Roll back or hold
The 403 persists with the new role
Observed caller is not the new principal
Network or authorization mode actually explains the denial
Moving to user-assigned identity changes an unvalidated contract

Do not run a tenant-wide cleanup of unknown identities during the incident. Remove assignments by exact ID after scope review and recovery validation.

Conclusion

An Azure resource name is not its security identity. After recreation, compare principalId values, prove the old and new role assignments, then confirm the caller in target logs before changing authorization.

The closing decision should be explicit: restore one minimum role to the new principal and repair the IaC dependency, roll back an uncontrolled recreation, or adopt a user-assigned identity when authorization must outlive compute. Recovery is complete when useful access works, out-of-contract access remains denied, and orphaned assignments can be removed without ambiguity.