Azure Private DNS: validate a zone change before breaking private paths

An Azure Private DNS change often looks harmless: add a record, move a zone to another resource group, link a new VNet, fix a suffix, adjust a forwarder or lower a TTL. In production, that change can make a private service invisible from part of the network, resolve a name to a public IP, break an internal agent or create timeouts that the team first attributes to the application.

The use case is an Azure environment with several spokes, App Service or Functions workloads using VNet Integration, operations VMs, private runners, a Private DNS Resolver and private zones used by Storage, Key Vault, SQL, Service Bus or internal APIs. The goal of the runbook is not to “fix DNS” by instinct. It must prove what will change, from which networks resolution must work, which cache can hide the real effect and how to return to the previous state if the private path breaks.

Describe the change as a resolution path

A private zone is not only a list of records. It is a complete path: client, local resolver, VNet link, private zone, optional forwarding, cache and then the called service. Start by writing that path down before touching the zone.

Requested change
Zone: privatelink.database.windows.net
Object: add or correct an A record for sql-prod-01
Affected environments: production, spoke-app, spoke-ops
Dependent workloads: orders API, billing job, private runner
Expected resolver: Azure Private DNS through VNet link or Private Resolver
Expected resolution: private IP in the Private Endpoint subnet

Questions before change
Which VNets must resolve this name?
Are VNet links automatic or manually managed?
Is an on-premises forwarder or Private Resolver involved?
Which TTL is currently observed by workloads?
Which application test proves the resolution is usable?

If the change involves a Private Endpoint, keep it in its proper role: it provides a private IP and an expected record, but the incident may come from a VNet link, resolver, cache, duplicated zone or forwarder. The Private Endpoint does not explain the whole path.

Build the validation matrix

Validating from Azure Portal or from an administration workstation is not enough. Resolution must be tested from the networks and identities that really run the workloads. The matrix prevents the team from confusing “the zone contains the right record” with “the service is reachable from production”.

Validation points
spoke-app / integration subnet / orders API
  Tested name: sql-prod-01.privatelink.database.windows.net
  Expected: private IP 10.42.18.7
  Test: resolution + controlled SQL connection

spoke-ops / diagnostic VM
  Tested name: sql-prod-01.database.windows.net
  Expected: CNAME to privatelink, then private IP
  Test: nslookup, dig, TCP connection 1433

private-runner / cicd subnet
  Tested name: sql-prod-01.database.windows.net
  Expected: same resolution as the workload
  Test: preflight before pipeline

on-premises / enterprise resolver
  Tested name: sql-prod-01.database.windows.net
  Expected: forward to Private Resolver, then private IP
  Test: dig with explicit resolver

The matrix should include at least one test for each real network path. If an application uses VNet Integration, test from a diagnostic container or job using the same VNet, not only from Cloud Shell.

Read Azure state before changing

Before the change, capture the state of zones, links and records. This evidence supports both diagnosis and rollback. It also avoids fixing the wrong zone when several private zones carry the same suffix across subscriptions.

az network private-dns zone show --resource-group rg-network-prod --name privatelink.database.windows.net --query "{id:id,name:name,numberOfRecordSets:numberOfRecordSets}" --output json

az network private-dns link vnet list --resource-group rg-network-prod --zone-name privatelink.database.windows.net --query "[].{name:name,vnet:virtualNetwork.id,registration:registrationEnabled,provisioningState:provisioningState}" --output table

az network private-dns record-set a list --resource-group rg-network-prod --zone-name privatelink.database.windows.net --query "[].{record:name,ttl:ttl,ips:aRecords[].ipv4Address}" --output table

Archive this output in the change ticket. A clean DNS rollback needs to know whether the record existed, which TTL it had, which VNets were linked and which subscription hosted the effective zone.

Test resolution from the workload

The useful test starts from the workload network. For App Service or Functions, that may mean Kudu console, a diagnostic container, a temporary job or a synthetic probe placed in the same spoke. For a private runner, the DNS preflight should run before deployment.

set -euo pipefail

HOST_PUBLIC="sql-prod-01.database.windows.net"
HOST_PRIVATE="sql-prod-01.privatelink.database.windows.net"
EXPECTED_PRIVATE_IP="10.42.18.7"

resolved_public="$(getent hosts "$HOST_PUBLIC" | awk '{print $1}' | head -1)"
resolved_private="$(getent hosts "$HOST_PRIVATE" | awk '{print $1}' | head -1)"

echo "public-name=$resolved_public"
echo "private-name=$resolved_private"

test "$resolved_private" = "$EXPECTED_PRIVATE_IP"
test "$resolved_public" = "$EXPECTED_PRIVATE_IP"

timeout 5 bash -c "cat < /dev/null > /dev/tcp/$HOST_PUBLIC/1433"

The test intentionally checks both the public service name and the privatelink name. For many Azure services, the application calls the public FQDN, which must be rewritten through CNAME and then resolved to the private address. Testing only the privatelink name can hide a broken full chain.

Watch caches and TTL

A DNS change may look successful from a fresh VM and still fail inside a workload that keeps a local cache, a connection pool or a previously resolved address. Treat TTL as a deployment constraint, not as a zone detail.

Caches to check
Local system or container resolver
Application cache or connection pool
Sidecar, proxy or internal gateway
Private Resolver and on-premises forwarder
Upstream enterprise DNS
Already warm workload instances

Window decision
Short TTL before cutover when possible
Validation before and after relevant TTL expiry
Application restart only when cache behavior justifies it
No rollback declared before testing from a fresh client and a warm client

If the cutover must be quick, lower the TTL before the change window and wait until it is really propagated. Otherwise, you may validate a transient behavior that disappears after caches expire.

Read logs as path evidence

DNS alone does not prove that the service is usable. After the change, correlate resolution, connection and service-side logs. For Azure services, diagnostic logs, connection metrics, application errors and network traces provide evidence that the private path works.

let ChangeWindowStart = datetime(2026-06-26 08:00:00);
let ChangeWindowEnd = datetime(2026-06-26 09:00:00);
AppTraces
| where TimeGenerated between (ChangeWindowStart .. ChangeWindowEnd)
| where Message has_any ("sql-prod-01", "database.windows.net", "connection")
| project TimeGenerated, AppRoleName, Message, SeverityLevel, OperationId
| order by TimeGenerated desc

Adapt the table to your tooling: Application Insights, AzureDiagnostics, firewall logs, runner traces or called API logs. The question stays the same: after private resolution, does the service answer through the right path and with the right identity?

Decide: deploy, fix or roll back

The change should be accepted only when the matrix passes from critical paths. A zone that is correct in Azure but unresolved from a spoke is an incomplete change.

Validate the change
Expected records exist in the right zone
Critical VNet links are present and Succeeded
Workloads resolve the public FQDN to the expected private IP
Application tests pass after relevant TTL expiry
Logs show no increase in timeouts or authentication errors

Fix without full rollback
A single VNet is missing a link
A forwarder points to the wrong resolver
A record has the right name but the wrong IP
The service is healthy from other paths

Roll back
Several critical spokes resolve to public or NXDOMAIN
The change touched an undocumented shared zone
Application tests fail after a quick correction
TTL makes the state unstable during the window
The previous record or link is known and restorable

DNS rollback must restore the previous state, then replay the same tests. Do not consider rollback complete until critical workloads have recovered the expected resolution and application errors stabilize.

Automate the control, not the exception

Good automation should not merely apply records. It should block dangerous changes: missing zone, missing VNet link, unexpected public record, IP outside private ranges, no test from the runner or divergence between environments.

guardrails:
before_apply:
  - capture_zone_state
  - verify_expected_vnet_links
  - verify_record_diff_is_scoped
  - run_dns_preflight_from_private_runner
block_when:
  - expected_vnet_link_missing
  - record_points_to_public_ip
  - zone_suffix_not_in_approved_list
  - rollback_state_not_captured
after_apply:
  - test_public_fqdn_resolves_private
  - test_private_fqdn_resolves_expected_ip
  - run_application_probe
  - attach_evidence_to_change_ticket

The goal is not to make DNS more bureaucratic. The goal is to make every change provable before a resolution incident becomes an application incident.

Conclusion

Azure Private DNS should be operated as a production path, not as a simple record inventory. A reliable change describes affected clients, VNet links, resolvers, caches, application tests and rollback before modification.

The healthy decision is simple: deploy when the matrix proves the private path from workloads, fix when the gap is localized, roll back when several critical paths become ambiguous. That is what lets private zones evolve without turning every DNS change into an incident diagnosis.