Azure UDR and NAT Gateway: diagnose egress before opening the firewall

An Azure egress failure often becomes a simple request: “open the firewall”. The symptom looks obvious. The application can no longer reach an external API, a registry, a SaaS service or an Azure endpoint. But adding a rule without diagnosis can hide a wrong UDR, an overloaded NAT Gateway, a restrictive NSG, unexpected DNS resolution or a changed source IP. The risk is making the path more permissive without fixing the cause.

The use case is an Azure workload integrated with a VNet. It reaches the Internet or a partner service through a route table, central firewall, NAT Gateway and sometimes a proxy. After a network deployment, outbound calls start failing. The runbook has one goal: decide whether to fix routing, adjust the firewall, stabilize NAT egress, roll back the network change or refuse a broad opening because the evidence is not strong enough.

Freeze the flow before changing rules

Before editing the firewall, name the exact flow. “The application lost Internet access” is too broad. Diagnosis should isolate the source, destination, port, protocol, FQDN and expected network path.

Flow to qualify
Workload and source subnet
Destination FQDN or IP
Port and protocol
Application identity or job involved
First failure time
Recent change: route table, NSG, NAT, firewall, DNS, proxy

Expected path
Source subnet
Applied NSG
Route table and next hop
Central firewall or NVA
NAT Gateway or public egress IP
Partner rule or external allowlist
Logs available for each hop

This framing prevents the team from fixing the wrong layer. The evidence is different depending on whether the destination is a public Azure service, a partner endpoint or a private service exposed elsewhere.

Separate DNS, routing and filtering

An HTTP or TLS failure does not prove that the firewall is blocking. Start by proving name resolution, then next hop, then filtering. All three layers can create the same application symptom.

TARGET_FQDN=api.partner.example.com
TARGET_PORT=443

echo "dns"
getent hosts "$TARGET_FQDN" || nslookup "$TARGET_FQDN"

echo "tcp"
timeout 5 bash -lc "cat </dev/null >/dev/tcp/$TARGET_FQDN/$TARGET_PORT" && echo "tcp_connect_ok=true" || echo "tcp_connect_ok=false"

echo "tls"
openssl s_client -connect "$TARGET_FQDN:$TARGET_PORT" -servername "$TARGET_FQDN" </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -dates

echo "observed egress ip"
curl -sS --connect-timeout 5 https://ifconfig.me || true

The observed source IP must match the design. If the egress IP changed, a partner allowlist may reject the flow even though Azure routing works. If DNS resolves to a different target than expected, the firewall rule may look missing while the flow is simply going to the wrong destination.

Read effective routes from the subnet

A UDR can send all traffic to a firewall, let some destinations go out directly, or create different behavior between environments. The control should start from the representative network interface or subnet, not only from the architecture diagram.

NIC_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-prod/providers/Microsoft.Network/networkInterfaces/app-nic01"

az network nic show-effective-route-table --ids "$NIC_ID" --output table

az network nic list-effective-nsg --ids "$NIC_ID" --output table

The question is not only whether a default route exists. Check the most specific prefix, the real next hop, the priority between system routes and user-defined routes, then the NSG behavior attached to the same path.

Check NAT Gateway without reducing the issue to SNAT

NAT Gateway stabilizes outbound IPs and increases SNAT capacity, but it does not fix a bad UDR or a firewall that rejects the flow. In this diagnosis, NAT must answer two questions: which egress IP is expected, and is there evidence of capacity pressure or path drift?

Check
NAT Gateway attached to the expected subnet
Expected public IP or prefix
No alternate path through another egress IP
External allowlist aligned with observed IP
Long connections or bursts compatible with SNAT capacity
Recent subnet, route table or public IP prefix change

Possible decision
Wrong egress IP: fix association or route before firewall
Probable exhaustion: reduce reuse, scale, add capacity or fix client behavior
NAT looks correct: continue with firewall, DNS, proxy or application diagnosis

A source IP change is often treated as a partner firewall outage. Sometimes it is. The durable fix is still to make egress explicit and verifiable, not to add every possible IP to an allowlist.

Correlate denied traffic in logs

If the expected path crosses Azure Firewall or another logged NVA, logs should confirm a denial, no flow, or an unexpected destination. KQL should stay scoped: source, destination, port, time window, action.

let Window = 2h;
let SourcePrefix = "10.42.";
let TargetFqdn = "api.partner.example.com";
AzureDiagnostics
| where TimeGenerated > ago(Window)
| where Category has_any ("AzureFirewallNetworkRule", "AzureFirewallApplicationRule", "AzureFirewallDnsProxy")
| extend msg = tostring(msg_s)
| where msg has TargetFqdn or msg has SourcePrefix
| project TimeGenerated,
        Category,
        OperationName,
        msg,
        Resource,
        ResourceGroup
| order by TimeGenerated desc

An empty result does not automatically justify an opening. It may mean the route does not cross the firewall, the enabled logs do not cover the right category, DNS does not resolve to the expected destination, or the flow fails before filtering.

Decide open, fix or roll back

The decision should separate four cases. Opening the firewall is acceptable only when the path, destination and denial are proven.

Open or adjust the firewall
Destination and port confirmed
Effective route to the firewall confirmed
Denial observed in logs
Rule scope limited to the application and target FQDN
Expiration or review planned for temporary exception

Fix routing
Unexpected next hop
More specific route applied incorrectly
Test and production differ
Firewall is bypassed although it should be crossed

Fix NAT or allowlist
Observed source IP differs from design
NAT Gateway missing from expected subnet
Partner allowlist does not match
Capacity pressure or public IP prefix change is visible

Roll back the network change
Several critical flows regress after the same route table change
The change widened or diverted traffic
Local correction is riskier than reverting
Previous path is known and testable

This matrix prevents the easy exception. A broad firewall rule may make the immediate test pass while leaving a bad UDR or uncontrolled egress path in place.

Automate the evidence pack

The diagnosis can be prepared by an AWX job, pipeline stage or internal tool. Automation should collect evidence, not decide alone that a network opening is safe.

evidence_pack:
flow:
  source_subnet: snet-app-prod
  target_fqdn: api.partner.example.com
  port: 443
required:
  - dns_result_from_workload_network
  - effective_route_table
  - effective_nsg
  - observed_egress_ip
  - firewall_or_proxy_logs
  - rollback_path
block_firewall_change_when:
  - destination_not_confirmed
  - route_does_not_cross_firewall
  - observed_egress_ip_unknown
  - no_owner_for_broad_exception
allow_human_review_when:
  - refused_flow_is_proven
  - scope_is_limited
  - validation_probe_exists
  - rollback_is_documented

The useful deliverable is not a red or green status. It is a reviewable packet that explains why a rule is needed, or why it is not.

Validate and keep rollback close

After the correction, replay the probe from the same network point. Also verify that nearby flows did not start using an unexpected path and that logs remain usable.

Validation after action
DNS matches the expected result
TCP and TLS work from the workload network
Egress IP matches the design
Firewall or proxy logs are visible
Application call succeeds with a correlation ID
No broad bypass was added without expiration

Rollback
Restore the previous route table
Remove the temporary firewall rule
Restore the expected NAT association
Replay the probe and compare before and after
Keep evidence in the incident or change ticket

If the test does not recover after the firewall opening, stop adding rules and return to the path: DNS, UDR, NSG, NAT, proxy, then application.

Conclusion

An Azure egress incident should not be solved by reflexively opening the firewall. The flow needs to be qualified end to end: name, destination, DNS, effective route, NSG, NAT Gateway, firewall, logs and application validation.

The right decision is sometimes a targeted rule. Often it is something else: fixing a UDR, stabilizing the outbound IP, rolling back a route table or rejecting an exception that is too broad. That discipline turns Azure egress into an operable path instead of a stack of exceptions that nobody can explain later.