Infrastructure

Azure Backup: validate a restore point before declaring rollback ready

A production runbook for qualifying an Azure Backup restore point with application consistency, dependencies, identity, networking, test restore, evidence, decision and rollback before cutover.

17 Jul 2026 azureazure-backupbackuprestorerollbackdisaster-recoveryidentitynetworkingobservabilityrunbookproduction

An Azure Backup restore point visible in the portal is not enough to declare rollback ready. It proves that a backup exists. It does not prove that the application can restart with consistent data, the right identities, expected network dependencies and a controlled backout path.

The use case is a production application affected by a failed migration, data corruption, destructive deployment or operational mistake. A team wants to restore a VM, disk, database or file share protected by Azure Backup. The runbook goal is to decide whether the restore point is usable, whether it should be restored into isolation, partially applied, held for more evidence or abandoned in favor of another rollback path.

Freeze the restore scope

Start by writing down what must roll back. A production rollback is not always a full restore. It may target a data disk, a VM, a database, a critical file, configuration state or a set of resources that must remain consistent together.

text restore-scope.txt

Incident
Affected service: api-orders-prod
Suspect change: schema migration + recovery batch
Impact window: 2026-07-17 06:20-07:10 UTC
Goal: restore consistent data without losing valid transactions

Candidate scope
Workload: VM, disk, Azure Files, SQL in VM or application data
Recovery Services vault: rsv-prod-core
Backup policy: prod-daily-hourly
Target restore point: before the suspect change
Dependencies: Key Vault, identity, DNS, firewall, storage, queue, downstream API

Decision to produce
Restore into isolation
Restore partially
Cut over to the restore
Replay or rebuild part of the data
Reject the point and use another rollback path

If the scope is not explicit, the team can restore a technically valid resource that is functionally inconsistent.

Check the actual restore point

The first control is restore point freshness, type and state. Do not pick a point only because it is the latest available one: it may be after the corruption, crash-consistent only, incomplete, or tied to a policy that does not cover every dependency.

bash 01-list-restore-points.sh

SUBSCRIPTION="00000000-0000-0000-0000-000000000000"
VAULT_RG="rg-prod-backup"
VAULT="rsv-prod-core"
CONTAINER="iaasvmcontainer;iaasvmcontainerv2;rg-prod-app;vm-orders-01"
ITEM="vm;iaasvmcontainerv2;rg-prod-app;vm-orders-01"

az account set --subscription "$SUBSCRIPTION"

az backup recoverypoint list --resource-group "$VAULT_RG" --vault-name "$VAULT" --container-name "$CONTAINER" --item-name "$ITEM" --backup-management-type AzureIaasVM --query "[].{time:recoveryPointTime,type:recoveryPointType,tier:recoveryPointTier,consistency:recoveryPointConsistencyType}" --output table

The selected point must connect to the incident timeline. A point that is too recent may reintroduce the fault. A point that is too old may lose transactions that could have been preserved another way.

Separate available backup from usable restore

An available backup is not yet a usable rollback. Test the restore in a scope that does not modify active production: temporary resource, isolated network, recovery subscription, stopped VM or restored disk without immediate attachment.

text restore-readiness-checks.txt

Evidence before cutover
Restore point is before the suspect change
Consistency type is acceptable for the workload
Test restore completes without error
Restored data passes a minimal application check
Required secrets and identities still exist
Routes, DNS and firewall allow testing without exposing production
Downstream dependencies cannot consume restore data by accident

Block
Restoring directly over the active resource without test
Attaching a restored disk to production without a write freeze
Running two copies of the same workload with the same identities and consumers
Replaying application jobs before idempotency is checked

This separation prevents a rollback that succeeds technically but creates a second write path, duplicate message consumption or identity collision.

Restore into an evidence zone

The evidence zone must allow inspection without business side effects. For a VM, restore disks or create an isolated VM. For files, restore to a temporary path. For a database hosted on a VM, restore offline before opening it to applications.

bash 02-restore-disks-for-validation.sh

TARGET_RG="rg-restore-validation"
STORAGE_ACCOUNT="strestoreevidence"
RESTORE_POINT_NAME="AzureBackup_20260717_052000"

az backup restore restore-disks --resource-group "$VAULT_RG" --vault-name "$VAULT" --container-name "$CONTAINER" --item-name "$ITEM" --rp-name "$RESTORE_POINT_NAME" --storage-account "$STORAGE_ACCOUNT" --target-resource-group "$TARGET_RG" --restore-to-staging-storage-account true

The exact command depends on workload type and enabled options. The stable principle is to produce an inspectable restored artifact, with limited permissions, before any cutover.

Validate application consistency

The useful test is not only whether the VM boots or files exist. It must prove that the application can read its data, find dependencies and refuse dangerous actions until cutover is approved.

text application-consistency-probe.txt

Minimum checks
Mount the disk or open the restored database read-only
Verify schema version, applied migrations and data timestamp
Compare a sample of business keys with pre-incident logs
List missing transactions between restore point and write freeze
Verify referenced secrets still exist in Key Vault
Verify runtime identity does not trigger jobs or webhooks
Test a representative application query without downstream writes

Rejection signals
Schema is incompatible with the currently deployed code
Restored data has already been compensated by another process
Queues, webhooks or batches can process duplicates
Restored identity has unbounded production permissions

For many incidents, the best decision is not to cut over to the full restore. It is to restore a subset, extract data, or use the restore as a comparison source.

Read Backup and Activity Log signals

Keep evidence from both backup and restore operations. It helps separate an unusable point, failed restore, permission problem, quota limit or network issue in the evidence zone.

kusto 03-backup-restore-evidence.kql

let StartTime = datetime(2026-07-17T05:00:00Z);
let EndTime = datetime(2026-07-17T08:00:00Z);
AzureActivity
| where TimeGenerated between (StartTime .. EndTime)
| where ResourceProviderValue has_any ("MICROSOFT.RECOVERYSERVICES", "MICROSOFT.COMPUTE", "MICROSOFT.STORAGE")
| where OperationNameValue has_any ("restore", "backup", "write", "delete", "disks")
| project TimeGenerated, ResourceGroup, ResourceProviderValue, OperationNameValue, ActivityStatusValue, Caller, CorrelationId, Properties
| order by TimeGenerated desc

Complete this with vault jobs, application logs and IaC changes. A rollback without reliable timestamps becomes hard to defend if data loss is discovered later.

Decide restore, extract or reject

The decision must account for business impact, data consistency, acceptable loss window and duplicate-write risk. It must also name the reverse plan: how to back out if the restore makes things worse.

yaml restore-decision.yml

decision:
restore_to_production:
  when:
    - restore_point_precedes_fault
    - application_consistency_validated
    - write_freeze_or_cutover_window_confirmed
    - dependencies_ready_for_restored_state
  validation:
    - health_probe_passes
    - business_sample_matches_expected_state
    - no_duplicate_consumers_running

partial_restore_or_extract:
  when:
    - only_some_records_or_files_are_needed
    - full_restore_would_lose_valid_work
    - restored_copy_is_safe_as_read_only_source
  validation:
    - extracted_items_reviewed
    - target_update_is_idempotent
    - audit_trace_keeps_source_restore_point

hold:
  when:
    - restore_point_consistency_is_unclear
    - downstream_side_effects_are_not_controlled
    - identity_or_network_path_is_not_isolated

abandon_restore_point:
  when:
    - point_is_after_corruption
    - schema_or_data_is_incompatible
    - safer_recovery_path_exists

A production restore must be a decision, not the automatic consequence of a successful backup.

Prepare cutover and rollback of the rollback

Before cutover, define the write freeze, active identities, consumers to stop, expected probes and the return point if the restore fails. Rollback of the rollback is rarely comfortable, but it must exist.

text cutover-and-backout.txt

Before cutover
Freeze writes or stop affected consumers
Back up current state before replacement
Name the restore point being used
Disable jobs that would replay twice
Verify DNS, routes, NSG and firewall for the restored resource
Verify identities, secrets and minimum permissions
Announce the window and stop criteria

Post-cutover validation
Application health check
Business sample read
Logs without identity, disk, database or network errors
No duplicate message processing
Azure Monitor alerts return to expected noise level

Backout
Stop the restored workload
Return to the pre-cutover resource or snapshot
Remove temporary permissions
Keep evidence artifacts and failure reason

If the plan cannot stop consumers or return to the pre-cutover state, the restore is not ready yet.

Conclusion

Azure Backup provides a starting point, not a rollback decision. The runbook must tie the restore point to the incident timeline, test restoration outside production, prove application consistency, control identities and dependencies, then decide between full cutover, partial restore, hold or rejection.

The right closing signal is not “the restore exists”. It is “we know what this point restores, what it loses, what it can break, how to validate it and how to back out if cutover fails”.