Failure Atlas

Start from the symptom, not the category.

A diagnostic map for operational situations: 502 errors, private DNS drift, WAF blocks, Terraform locks, automation guardrails and private AI controls.

Cloud 3 notes

Internal APIM returns an error on a private API

Correlate Application Gateway/WAF and APIM logs, then separate DNS, TLS, policy, identity and private backend reachability before changing policies or opening access.

First checks
  • Check whether WAF blocked the request
  • Confirm APIM received the same path
  • Validate backend DNS and TLS from the APIM path
  • Replay with a correlation ID
  1. 01 Azure internal APIM: diagnose a private API before changing policies
  2. 02 KQL snippet: correlate WAF and APIM on an Azure private API
  3. 03 Azure Application Gateway: diagnose 502 errors without mixing DNS, TLS and backend health
Networking 4 notes

Private Endpoint name still resolves publicly

Confirm the CNAME chain, Private DNS Zone association and hybrid forwarding from the consuming network.

First checks
  • Run nslookup from the workload network
  • Check privatelink CNAME
  • Verify Private DNS Zone links and forwarders
  1. 01 Azure snippet: check Private Endpoint DNS resolution
  2. 02 Azure snippet: detect Private Endpoint DNS drift
  3. 03 Azure hybrid DNS: when to use Private Resolver, on-premises forwarders and private zones
  4. 04 Azure Private Endpoint: detect Terraform, DNS, and network drift before incident
Cloud 3 notes

Azure Storage private endpoint returns 403, times out or produces no request logs

Separate Storage subresource DNS, Private Endpoint approval, firewall rules, runtime identity and Storage logs before opening public access or broadening RBAC.

First checks
  • Resolve the exact Storage subresource from the workload network
  • Check Private Endpoint status and private DNS zone group
  • Replay with a client request ID
  • Correlate Storage logs for 403, caller IP and requester identity
  1. 01 Azure Storage: diagnose a private endpoint without opening the account
  2. 02 KQL snippet: isolate Azure Storage 403 on a private endpoint
  3. 03 Azure Private Endpoint: detect Terraform, DNS, and network drift before incident
Cloud 3 notes

Azure SQL private endpoint returns timeouts, firewall errors or no SQL logs

Separate SQL private DNS, Private Endpoint state, public access, firewall rules, runtime identity and SQL diagnostics before changing schema, code or broad permissions.

First checks
  • Resolve the SQL FQDN from the workload network
  • Check Private Endpoint status and privatelink.database.windows.net records
  • Replay with the real runtime identity
  • Correlate SQL diagnostics for firewall and login errors
  1. 01 Azure SQL: diagnose a private endpoint before changing the database
  2. 02 KQL snippet: isolate Azure SQL private endpoint errors
  3. 03 Azure Private Endpoint: detect Terraform, DNS, and network drift before incident
Cloud 3 notes

Azure Service Bus private endpoint times out, denies access or lets backlog grow

Separate Service Bus private DNS, Private Endpoint state, public access, managed identity or SAS, queue metrics and processing logs before touching queues or redeploying consumers.

First checks
  • Resolve the Service Bus FQDN from the workload network
  • Check Private Endpoint approval and public network access
  • Verify the real sender or receiver identity
  • Correlate backlog, dead-letter and Service Bus errors
  1. 01 Azure Service Bus: diagnose a private endpoint before touching queues
  2. 02 KQL snippet: isolate Service Bus private endpoint errors
  3. 03 Azure Private Endpoint: detect Terraform, DNS, and network drift before incident
Cloud 4 notes

A synthetic probe fails on an Azure private path

Separate DNS, TLS, Application Gateway health, WAF blocks and runner network before changing routing or application code.

First checks
  • Resolve the hostname from the probe network
  • Check TLS/SNI with the real hostname
  • Correlate probe run with WAF and gateway logs
  1. 01 Azure: make private paths verifiable with synthetic probes
  2. 02 KQL snippet: track synthetic probes for an Azure private path
  3. 03 Azure Application Gateway: diagnose 502 errors without mixing DNS, TLS and backend health
  4. 04 Azure Private Endpoint: detect Terraform, DNS, and network drift before incident
Networking 4 notes

Azure traffic leaves through one path and returns through another

Compare DNS target, effective routes, route table associations, firewall evidence, NAT identity and return path before changing UDRs or bypassing inspection.

First checks
  • Resolve the destination from the source path
  • Compare effective routes on source and destination NICs
  • Check firewall or appliance logs for both directions
  • Confirm NAT or outbound source identity
  1. 01 Azure route asymmetry: diagnose before changing a UDR
  2. 02 Azure NSG, UDR and Firewall: diagnose blocked traffic before adding a rule
  3. 03 Azure Firewall: diagnose a shadowed rule before opening traffic
  4. 04 Azure UDR and NAT Gateway: diagnose egress before opening the firewall
Cloud 3 notes

Azure Container Apps private ingress fails or reaches the wrong revision

Separate private DNS, Application Gateway handoff, Container Apps ingress mode, revision traffic and console logs before rolling back or changing traffic weights.

First checks
  • Resolve the hostname from the caller network
  • Check ingress target port and active revisions
  • Correlate system and console logs
  1. 01 Azure Container Apps: diagnose private ingress before changing revisions
  2. 02 KQL snippet: diagnose Container Apps private ingress and revisions
  3. 03 Azure: make private paths verifiable with synthetic probes
Cloud 3 notes

AKS private ingress returns 502 or reaches no service endpoints

Separate private DNS, Application Gateway health, ingress controller routing, Kubernetes service selectors, endpoint slices and pod readiness before rolling back a deployment.

First checks
  • Resolve the hostname from the caller network
  • Check Application Gateway backend health and host header
  • Verify ingress, service and endpoint slices
  • Correlate controller and application logs
  1. 01 Azure AKS: diagnose private ingress before changing deployments
  2. 02 KQL snippet: correlate AKS private ingress and application logs
  3. 03 Azure Application Gateway: diagnose 502 errors without mixing DNS, TLS and backend health
Infrastructure 1 note

An AKS node pool upgrade is blocked while draining a pod protected by a PDB

Separate unavailable replicas, readiness failures, pod placement, surge capacity and an over-constrained PodDisruptionBudget before deleting the safeguard or forcing the upgrade.

First checks
  • Capture the blocked node and eviction events
  • Inspect PDB disruptionsAllowed and selectors
  • Verify replacement capacity, replica readiness and topology
  • Resume only with positive disruption headroom
  1. 01 Azure AKS: unblock a node pool upgrade stopped by a PDB
Cloud 3 notes

Azure Functions private HTTP endpoint returns 403, 503 or no request logs

Separate private DNS, Private Endpoint reachability, access restrictions, Functions runtime state, private storage and Application Insights evidence before redeploying code or opening public access.

First checks
  • Resolve the hostname from the caller network
  • Replay with a correlation ID
  • Check Function App access restrictions and Private Endpoint status
  • Correlate requests, traces and exceptions
  1. 01 Azure Functions: diagnose a private HTTP endpoint before changing code
  2. 02 KQL snippet: correlate an Azure Functions private HTTP endpoint
  3. 03 Azure: make private paths verifiable with synthetic probes
Cloud 5 notes

Azure WAF blocks a legitimate request

Start from blocked requests, rule ID and URI before deciding between exclusion, custom rule or application fix.

First checks
  • List blocked URIs in KQL
  • Identify ruleId and match field
  • Validate false-positive scope
  1. 01 KQL snippet: list Azure WAF blocked URIs quickly
  2. 02 WAF and KQL: identify a false positive before creating an exclusion
  3. 03 Azure WAF: add an OWASP/CRS exclusion without weakening all protection
  4. 04 Azure WAF: frame an emergency custom rule without losing evidence
  5. 05 Azure snippet: audit WAF custom rule priorities
Infrastructure 2 notes

Azure Monitor fires an alert storm after deployment

Separate real service impact, noisy dimensions, threshold drift, action group behavior and rollback before silencing notifications or changing rules.

First checks
  • Group fired alerts by rule and target
  • Compare first alert time with the deployment window
  • Check service logs before changing thresholds
  • Keep one user-symptom alert active
  1. 01 Azure Monitor: diagnose an alert storm after deployment
  2. 02 Monitoring: turn an alert into an actionable operations runbook
Infrastructure 5 notes

A secret rotation, federated identity or managed identity change breaks an application or pipeline consumer

Separate preparation, cutover, revocation, workload identity federation and managed identity diagnostics; validate the real execution identity, private path and authentication errors before deleting the old value or broadening access.

First checks
  • List real consumers
  • Verify the runtime identity and vault read access
  • Check OIDC claims or private DNS depending on the path
  • Watch 401/403/500, sign-in failures or Key Vault denials
  1. 01 Service identity and secret rotation: a production runbook, not an isolated task
  2. 02 KQL snippet: detect authentication errors after secret rotation
  3. 03 Azure managed identity: diagnose private access before changing permissions
  4. 04 KQL snippet: diagnose Key Vault denial with managed identity
  5. 05 Azure Workload Identity Federation: diagnose CI authentication before bringing back a secret
Automation 1 note

An Azure Functions poison queue keeps growing

Separate contract failures, transient dependencies, identity, resource pressure and partial side effects before replaying Storage Queue messages.

First checks
  • Peek a redacted sample without consuming messages
  • Correlate message IDs with invocation and dependency logs
  • Verify downstream state and idempotency before replay
  1. 01 Azure Functions: diagnose a poison queue before replaying messages