AgentOps: diagnose MCP tool shadowing before a production action
A production runbook for detecting MCP tool collisions, proving which tool the agent actually selected and validating a canary catalog before any sensitive action.
Read article
Tag
kql
121 articles connected to this technical signal.
AgentOps: contain sensitive data in agent traces before disabling observability
A production runbook for bounding a leak in agent traces, locating the unsafe field, canarying redaction and restoring useful observability.
Read articleAzure WAF: diagnose inconsistent policy enforcement after deployment
A production runbook for separating ARM rollout, WAF associations, rule priority and request variance before a global Application Gateway policy rollback.
Read articleAzure WAF Bot Manager: qualify a bot before adding an allow rule
A production runbook for separating legitimate automation, spoofed identities and hostile bot traffic before changing Azure WAF Bot Manager actions.
Read articleMicrosoft Sentinel: validate a watchlist before it drives automated response
A production runbook for proving watchlist freshness, schema, SearchKey, deletions and KQL behavior before allowing it to influence automated response.
Read articleAzure DevOps WIF: diagnose issuer or subject drift before bringing back a secret
A production runbook for comparing the service connection, federated credential, issuer, subject, audience and RBAC before repairing, migrating or rolling back an Azure DevOps identity.
Read articleAzure Monitor: contain log alert fan-out before changing the KQL rule
A production runbook for measuring split-by dimension cardinality, containing notification noise, canarying stable grouping and rolling back an Azure Monitor log alert without losing coverage.
Read articleAzure Firewall IDPS: validate a signature before switching it to deny
A production runbook for qualifying an Azure Firewall IDPS signature in alert mode, measuring impact, canarying deny and retaining a targeted rollback.
Read articleAzure Virtual WAN: validate Routing Intent before enforcing production inspection
A production runbook for qualifying Azure Virtual WAN Routing Intent across private flows, internet egress, effective routes, Azure Firewall, canaries, validation and rollback.
Read articleOpenTelemetry Collector: validate tail sampling before losing incident traces
A production runbook for qualifying OpenTelemetry tail sampling across trace affinity, late spans, memory pressure, policy coverage, Azure Monitor evidence, canary rollout and rollback.
Read articleMicrosoft Sentinel: diagnose a silent scheduled detection before widening its lookback
A production runbook for separating source ingestion, latency, analytics-rule execution, KQL windows and incident creation before changing a Microsoft Sentinel detection.
Read articleAzure Container Apps: diagnose a stuck revision before redeploying
A production runbook for separating image pull, startup, probes, configuration, secrets, resources and platform state when a Container Apps revision never becomes healthy.
Read articleAzure Container Registry: diagnose incomplete image replication before retrying a multi-region rollout
A production runbook for separating ACR replication, regional routing, digests, identity and network paths before retrying or rolling back a container rollout.
Read articleAzure Traffic Manager: diagnose a degraded profile before forcing failover
A production runbook for separating application health, probe contract, endpoint state, DNS answers, TTL and secondary capacity before forcing Traffic Manager failover.
Read articleAzure Functions Flex Consumption: diagnose stalled scale-out before raising the instance ceiling
A production runbook for separating demand, concurrency, scale groups, the instance ceiling, regional quota and downstream saturation before changing Function App capacity.
Read articleAzure APIM: diagnose backend TLS before disabling certificate validation
A production runbook for separating trust chain, hostname, SNI, expiry, TLS protocol and APIM configuration before disabling backend certificate validation.
Read articleAzure WAF: diagnose a file upload before raising size limits
A production runbook for separating file upload limits, request body size, inspection depth, WAF rules and backend rejection before a reversible change.
Read articleAzure Front Door: diagnose an unhealthy origin before changing routing
A production runbook to separate health probes, DNS/TLS, Private Link, backend capacity and routing before draining, failing over or rolling back.
Read articleAzure Managed Redis: contain a connection storm before scaling or failing over
A production runbook to separate client reconnects, server pressure, DNS/TLS and network saturation, then validate a fix, scaling decision or rollback.
Read articleAzure App Configuration: diagnose feature flag drift before rollback
A production runbook for qualifying inconsistent feature flag evaluations across instances by separating version, refresh, cache, targeting, telemetry and rollback.
Read articleAzure Logic Apps: contain connector throttling before adding retries
A production runbook for separating Logic Apps limits, connector throttling and downstream saturation, then reducing concurrency, validating a canary or rolling back without amplifying 429 responses.
Read articleAzure Network Watcher: migrate NSG flow logs without losing network evidence
A production runbook for inventorying NSG flow logs, enabling virtual network flow logs in parallel, validating the new KQL schema, and cutting over with an explicit rollback.
Read articleAzure API Management: contain a retry storm before scaling the backend
A production runbook for proving retry amplification in Azure API Management, protecting an overloaded backend, and choosing a bounded policy change, circuit breaker or rollback.
Read articleAzure Key Vault: recover a deleted secret before recreating it
A production runbook to qualify a Key Vault secret deletion, recover the soft-deleted object, validate its consumers and decide whether to keep or roll back recovery without exposing the value.
Read articleAzure Monitor DCR: diagnose missing logs before rolling back a transformation
A production runbook for qualifying missing logs after a Data Collection Rule change, separating source, stream, KQL transformation and destination, then validating or rolling back with a canary.
Read articleAzure Container Apps with Dapr: diagnose service invocation before redeploying
A production runbook for separating app ID, Dapr sidecars, application port, mTLS, resiliency and revisions when service invocation fails in Azure Container Apps.
Read articleAzure SQL: diagnose connection pool exhaustion before scaling
A production runbook to separate connection leaks, SQL saturation, network latency and identity renewal before increasing the service tier.
Read articleAzure Event Hubs: diagnose consumer lag before scaling or replaying
A production runbook to separate ingestion pressure, throttling, hot partitions, consumer failures and checkpoint drift before adding capacity or replaying events.
Read articleAzure Monitor Private Link: diagnose ingestion loss before reopening public access
A production runbook to isolate AMPLS, DCE, DCR, DNS, associations and Azure Monitor Agent when telemetry disappears after network hardening, then validate or roll back without broadly reopening public access.
Read articleAzure Resource Graph: reconstruct configuration drift before rollback
A production runbook for finding an Azure change with Resource Graph Change Analysis, connecting it to its actor and impact, then validating or rolling it back without reverting blindly.
Read articleAzure Firewall: diagnose a shadowed rule before opening traffic
A production runbook for proving which Azure Firewall rule handles a flow, separating routing from policy order, and applying a targeted fix with validation and rollback.
Read articleAzure Monitor: validate an SLO burn-rate alert before paging on-call
A production runbook for building and qualifying an Azure Monitor burn-rate alert by separating the SLI, error budget, short and long windows, telemetry quality, notification, validation and rollback.
Read articleAzure Functions: diagnose a poison queue before replaying messages
A production runbook for qualifying poisoned Queue trigger messages, separating deterministic and transient failures, proving idempotency, then replaying or rolling back without duplicating business side effects.
Read articleAgentOps: contain an AI agent consumption runaway before shutting down production
A production runbook for attributing token and tool-call growth, isolating amplification, enforcing an execution budget, and validating or rolling back an AI agent release.
Read articleAzure Monitor: roll out a DCR transformation without losing telemetry
A production runbook for canarying a KQL transformation in an Azure Monitor Data Collection Rule, comparing volume and schema, detecting rejected records, then validating or rolling back without an observability gap.
Read articleAzure Service Bus: repair a broken distributed trace before redeploying
A production runbook for recovering Application Insights and OpenTelemetry correlation across Azure Service Bus, qualifying W3C propagation, sampling and message processing, then validating or rolling back the fix.
Read articleAzure Container Apps Jobs: diagnose a KEDA scale rule before rerunning workers
A production runbook for qualifying an Azure Container Apps Job that no longer consumes correctly with KEDA scale rule, backlog, managed identity, secrets, logs, idempotency, validation and rollback before rerunning workers.
Read articleAzure Network Watcher: validate Flow Logs before opening an NSG rule
A production runbook for qualifying a blocked Azure flow with Network Watcher, NSG Flow Logs, effective security rules, routes, firewall, KQL evidence, validation and rollback before opening a broad rule.
Read articleAzure Firewall: validate Threat Intelligence before blocking production
A production runbook for qualifying Azure Firewall Threat Intelligence with logs, false positives, critical flows, targeted exceptions, deny decision and rollback before enabling Deny.
Read articleAzure APIM: diagnose a policy regression before backend rollback
A production runbook for qualifying an Azure API Management regression with APIM traces, policy diff, headers, cache, backend, identity, logs, validation and rollback before changing the API or target service.
Read articleAzure WAF: validate a managed rule upgrade before switching to Prevention
A production runbook for qualifying an Azure WAF OWASP/CRS managed rule upgrade with logs, false positives, Detection mode, policy diff, application validation, Prevention decision and rollback.
Read articleAzure Durable Functions: diagnose an orchestration before replay or purge
A production runbook for qualifying a stuck Azure Durable Functions orchestration with instance history, activity state, storage, idempotency, KQL, validation and rollback before replay, terminate or purge.
Read articleAzure Container Apps: diagnose Managed Identity before widening a role
A production runbook for qualifying Azure Container Apps 401/403 failures with Managed Identity, token evidence, RBAC scope, Key Vault, ACR, logs, validation and rollback before widening permissions.
Read articleMicrosoft Entra Workload ID: diagnose Conditional Access before excluding a CI pipeline
A production runbook for qualifying a Conditional Access block on a workload identity with service principal logs, policy scope, risk, CI identity, KQL evidence, bounded exception and rollback.
Read articleAzure Functions: diagnose a Timer Trigger before replaying a job
A production runbook for qualifying a failed Azure Functions Timer Trigger with execution history, schedule locks, storage, Application Insights, idempotency, validation and rollback before manual replay.
Read articleAzure Workbooks: diagnose observability drift before changing alerts
A production runbook for qualifying Azure Workbooks drift with KQL, Log Analytics, Application Insights, dimensions, ingestion latency, validation and rollback before changing alerts or dashboards.
Read articleAzure Front Door: qualify origin health before failing over
A production runbook for qualifying Azure Front Door origin health with probes, routing, DNS, TLS, WAF evidence, backend logs, validation and rollback before forcing failover.
Read articleAzure WAF: diagnose rate limiting before increasing the threshold
A production runbook for qualifying Azure WAF rate limiting with logs, counting key, real clients, false positives, load checks, threshold decision and rollback.
Read articleAzure APIM: diagnose mTLS client certificate failures before changing policies
A production runbook for qualifying Azure API Management mTLS failures with client certificate chain, hostname, policy, gateway logs, backend identity, validation and rollback before changing API policies.
Read articleAzure Application Insights: diagnose sampling before changing alerts
A production runbook for qualifying apparent Application Insights telemetry loss with sampling, ingestion, SDK configuration, KQL, alerts, validation and rollback before changing thresholds.
Read articleAzure Service Health: qualify a regional signal before failover
A production runbook for qualifying an Azure Service Health or Resource Health signal with user impact, dependencies, routing, DNS, observability, failover decision and rollback.
Read articleAzure Monitor: diagnose an Action Group before silencing alerts
A production runbook for qualifying an Azure Monitor Action Group by separating rules, receivers, webhooks, escalation paths, KQL evidence, validation and rollback before reducing notifications.
Read articleAzure App Service: diagnose a Managed Identity token failure before bringing back a client secret
A production runbook for qualifying an Azure App Service Managed Identity failure with IMDS endpoint, Entra ID, RBAC, Key Vault or target API evidence, logs, validation and rollback before reintroducing a client secret.
Read articleAzure Firewall: qualify a broad rule before tightening production access
A production runbook for tightening an Azure Firewall rule without breaking useful traffic, with inventory, KQL evidence, dependencies, controlled validation and rollback.
Read articleAzure APIM: diagnose a subscription key before rotating it in production
A production runbook for qualifying Azure API Management 401/403 errors with subscription key, product, API, consumer, logs, validation and rollback before rotation.
Read articleAzure Monitor: diagnose ingestion latency before changing a KQL alert
A production runbook for qualifying a delayed Azure Monitor alert by separating application timestamps, Log Analytics ingestion, KQL query windows, evaluation frequency, action groups, validation and rollback.
Read articleAzure Logic Apps: diagnose a workflow before resubmitting the run
A production runbook for qualifying a failed Azure Logic Apps execution with trigger, connectors, managed identity, payload, logs, validation and rollback before resubmit.
Read articleAzure Cosmos DB: diagnose 429 throttling before increasing RU/s
A production runbook for qualifying Cosmos DB 429 throttling with RU consumption, hot partitions, query shape, SDK retries, indexing, KQL evidence, scaling decision and rollback.
Read articleAzure Key Vault: diagnose latency and throttling before rotating secrets
A production runbook for qualifying Key Vault degradation by separating latency, throttling, identity, network path, application cache, logs and rollback before starting a secret rotation.
Read articleAzure Load Balancer: diagnose a health probe before changing the backend pool
A production runbook for qualifying an unhealthy Azure Load Balancer backend with probe behavior, NSGs, routing, logs, pool configuration, validation and rollback before changing the rule or redeploying.
Read articleAzure Application Gateway: validate a Key Vault certificate before rotation
A production runbook for qualifying an Application Gateway TLS certificate rotation from Key Vault with managed identity, listener checks, WAF evidence, probes, logs, validation and rollback.
Read articleAzure Managed Grafana: diagnose a dashboard or alert before changing KQL
A production runbook for qualifying an Azure Managed Grafana dashboard or alert with Azure Monitor datasource, managed identity, Log Analytics permissions, variables, traces, validation and rollback before changing KQL queries.
Read articleAzure Front Door: diagnose an origin before changing WAF or routing
A production runbook for qualifying an Azure Front Door failure with origin groups, health probes, TLS, Private Link, DNS, WAF, logs, validation and rollback before changing routing.
Read articleAzure Event Grid: diagnose dead-lettered events before replay
A production runbook for qualifying Azure Event Grid delivery failures with subscription filters, endpoint health, dead-letter storage, diagnostics, replay scope, validation and rollback before reprocessing events.
Read articleAzure Key Vault: diagnose secret references before rotation
A production runbook for qualifying a Key Vault reference failure with managed identity, RBAC, secret version, diagnostics, KQL, validation and rollback before rotating or broadening access.
Read articleAzure Monitor Action Groups: diagnose missing notifications before changing alerts
A production runbook for qualifying an Azure Monitor alert that fired but did not notify anyone, with action groups, receivers, processing rules, webhooks, evidence, validation and rollback.
Read articleAzure route asymmetry: diagnose before changing a UDR
A production runbook for qualifying Azure asymmetric routing with effective routes, UDRs, Azure Firewall, NAT Gateway, flow logs, validation and rollback before changing route tables.
Read articleAzure Automation: diagnose a runbook before rerunning the job
A production runbook for qualifying an Azure Automation failure with managed identity, parameters, modules, Hybrid Worker, webhooks, logs, validation and rollback before rerun.
Read articleAzure Automation: validate a scheduled runbook before it changes production
A production runbook for qualifying a scheduled Azure Automation job with trigger, managed identity, parameters, dry-run, logs, validation and rollback before allowing real action.
Read articleAzure APIM: validate a Named Values rotation before breaking a private API
A production runbook for qualifying an APIM secret rotation with Named Values, Key Vault, managed identity, private backend, logs, application validation and rollback.
Read articleAzure App Configuration: diagnose a feature flag before rollback
A production runbook for qualifying an Azure App Configuration or feature flag regression with labels, identities, refresh, Key Vault references, logs, validation and rollback.
Read articleAzure App Service slots: validate a swap before promoting to production
A production runbook for qualifying an Azure App Service slot swap with configuration drift, warmup, identity, dependencies, logs, validation and rollback before promotion.
Read articleAzure Firewall DNS proxy: diagnose egress before opening rules
A production runbook for qualifying an Azure egress failure with Firewall DNS proxy, FQDN resolution, UDRs, application rules, logs, validation and rollback before broadening traffic.
Read articleAzure Container Registry: diagnose an image pull failure before redeploying
A production runbook for qualifying an ACR image pull failure with identity, network, DNS, firewall, logs, node cache, validation and rollback before rerunning deployment.
Read articleAzure Monitor: diagnose missing logs before changing alerts
A production runbook for qualifying missing Azure Monitor logs with Diagnostic Settings, DCRs, ingestion, KQL, cost controls, validation and rollback before changing alerts.
Read articleAzure RBAC: diagnose authorization drift before widening a role
A production runbook for qualifying Azure RBAC drift with the real identity, scope, PIM, role assignments, deny assignments, logs, validation and rollback before adding permissions.
Read articleAzure Workload Identity Federation: diagnose CI authentication before bringing back a secret
A production runbook for qualifying an Azure OIDC authentication failure in a CI pipeline with federated credentials, claims, roles, logs, validation and rollback.
Read articleKQL snippet: find risky tool actions from an AI agent
A short query to isolate write-capable tool calls, their execution identity and correlation before disabling an AI agent.
Read articleAzure UDR and NAT Gateway: diagnose egress before opening the firewall
A production runbook for qualifying Azure egress failures with UDR, NAT Gateway, NSG, firewall, DNS, KQL, validation and rollback before adding an overbroad rule.
Read articleAzure WAF: prepare an evidence pack before a policy PR
A production runbook for documenting an Azure WAF change with KQL evidence, policy diff, application probes, security validation, merge decision and rollback.
Read articleKQL snippet: prepare evidence for a managed WAF rule exclusion
A short query to isolate rule, field, URI and examples before proposing a targeted WAF exclusion.
Read articleAgentOps: diagnose an AI agent action before rollback
A production runbook for qualifying an AI agent action with traces, sources, tools, identity, KQL, human validation and rollback without disabling the whole assistant.
Read articleAzure deployment from a private runner: diagnose the pipeline before rerun
A production runbook for qualifying an Azure deployment failure from a private runner by separating identity, DNS, routing, NSG, NAT, dependency access, logs and rollback.
Read articleKQL snippet: qualify a Storage Queue backlog on a private path
A short query to separate missing consumption, retries, poison messages and access errors on a private Storage Queue.
Read articleAzure Container Apps: diagnose outbound egress before changing code
A production runbook for qualifying Azure Container Apps outbound failures by separating DNS, UDR, NSG, NAT Gateway, firewall, identity, logs and rollback before changing the application.
Read articleAzure Monitor and KQL: decide a deployment rollback without silencing alerts
A production runbook for deciding an Azure rollback after deployment with Azure Monitor, KQL, impact correlation, regression evidence, validation and controlled recovery.
Read articleKQL snippet: diagnose Workload Identity Federation failures in CI
A short query to find federated principal sign-in failures before falling back to a client secret.
Read articleAzure WAF: move a policy from Detection to Prevention without breaking traffic
A production runbook for qualifying an Azure WAF policy before switching to Prevention with KQL evidence, change scope, application validation, rollback window and an operable decision.
Read articleKQL snippet: spot Azure Diagnostic Settings drift
A short query to find writes, deletes and destination changes that explain a sudden disappearance of logs.
Read articleAzure NSG, UDR and Firewall: diagnose blocked traffic before adding a rule
A production runbook for qualifying blocked Azure traffic by separating NSG, UDR, Azure Firewall, DNS, effective routes, logs and rollback before opening access too broadly.
Read articleKQL snippet: correlate APIM errors on a private API
A short query to separate subscription or key issues, private backend routing, APIM policy failure and application-side API errors.
Read articleAzure Monitor: diagnose an alert storm after deployment
A production runbook for qualifying an Azure Monitor alert storm after deployment by separating real signal, noise, regression, threshold drift, action group behavior and rollback.
Read articleKQL snippet: qualify an Azure Firewall denial on private egress
A short query to separate application rule denial, network rule denial and DNS proxy resolution before adding an overbroad rule.
Read articleAzure NAT Gateway: diagnose SNAT exhaustion and outbound IP drift
An operational runbook for qualifying Azure outbound incidents involving NAT Gateway, SNAT ports, UDR, NSG, firewalls, allowlists and rollback.
Read articleKQL snippet: isolate Application Gateway 502s to a private backend
A short query to separate unavailable backend, application failure and WAF blocking when Application Gateway returns 502s on a private path.
Read articleAzure Service Bus: diagnose a private endpoint before touching queues
An operational runbook for Azure Service Bus private access incidents by separating DNS, Private Endpoint, identity, firewall, metrics, logs and rollback.
Read articleKQL snippet: isolate Service Bus private endpoint errors
A short query to separate missing connection evidence, identity denial, expired SAS and processing errors during a private Service Bus incident.
Read articleAzure SQL: diagnose a private endpoint before changing the database
An operational runbook for Azure SQL private access failures by separating DNS, Private Endpoint, firewall, identity, SQL logs and rollback evidence.
Read articleKQL snippet: isolate Azure SQL private endpoint errors
A short query to separate firewall, authentication, public endpoint and missing connection evidence during a private Azure SQL incident.
Read articleAzure Storage: diagnose a private endpoint without opening the account
An operational runbook for Azure Storage private access failures by separating DNS, Private Endpoint, firewall, identity, logs and rollback evidence.
Read articleKQL snippet: isolate Azure Storage 403 on a private endpoint
A short query to separate identity, firewall, public endpoint and wrong subresource when private Azure Storage access is denied.
Read articleAzure App Service: diagnose a private endpoint before redeploying
Build an operational runbook for App Service private access failures by separating DNS, Private Endpoint, access restrictions, Application Gateway, application logs and rollback evidence.
Read articleKQL snippet: correlate private App Service and Application Gateway
A short query to separate WAF, gateway, private DNS, access restrictions and App Service logs during a private access incident.
Read articleAzure Functions: diagnose a private HTTP endpoint before changing code
Build an operational runbook for private Azure Functions failures by separating DNS, Private Endpoint, access restrictions, private storage, Application Insights logs and rollback evidence.
Read articleKQL snippet: correlate an Azure Functions private HTTP endpoint
A short query to separate DNS, private access, Functions runtime and application exceptions during a private HTTP incident.
Read articleAzure AKS: diagnose private ingress before changing deployments
Build an operational runbook for AKS private ingress failures by separating DNS, Application Gateway, ingress controller, Kubernetes service endpoints, pod readiness and rollback evidence.
Read articleKQL snippet: correlate AKS private ingress and application logs
A short query to read ingress controller and application logs together when a private AKS route returns 502, timeouts or no endpoints.
Read articleAzure Container Apps: diagnose private ingress before changing revisions
Build an operational runbook for Azure Container Apps private ingress failures by separating DNS, ingress mode, revision routing, application logs and rollback evidence.
Read articleKQL snippet: diagnose Container Apps private ingress and revisions
A short query to correlate Azure Container Apps system and console logs when private ingress, probes or revision traffic fail.
Read articleAzure internal APIM: diagnose a private API before changing policies
Qualify a failure across Application Gateway, WAF, internal APIM and a private backend by separating DNS, routing, policy, identity and logs before any fix.
Read articleKQL snippet: correlate WAF and APIM on an Azure private API
A short query to see whether a private API request is blocked by Application Gateway WAF, received by APIM or missing from the expected path.
Read articleAzure managed identity: diagnose private access before changing permissions
Build a runbook for Key Vault, Storage or private API access failures with managed identity, RBAC, private DNS, logs and real execution evidence.
Read articleKQL snippet: diagnose Key Vault denial with managed identity
A short query to separate identity denial, network path and source address when an Azure workload can no longer access Key Vault.
Read articleAzure: make private paths verifiable with synthetic probes
Build useful synthetic probes for DNS, TLS, Application Gateway, WAF and Private Endpoint so private Azure paths fail with evidence before production incidents.
Read articleKQL snippet: track synthetic probes for an Azure private path
A short query to track synthetic probe failures and separate DNS, TLS, WAF or Application Gateway symptoms on an Azure private path.
Read articleAzure WAF: frame an emergency custom rule without losing evidence
Apply a temporary Azure WAF custom rule with priority, KQL evidence, business validation and rollback, without permanently hiding managed-rule signals.
Read articleKQL snippet: detect authentication errors after secret rotation
A short query to watch 401, 403, and 500 errors after rotating an application secret or service identity.
Read articleKQL snippet: list Azure WAF blocked URIs quickly
A short KQL query to identify the most blocked URIs by Azure Web Application Firewall on Application Gateway.
Read articleAzure WAF: qualify a false positive before creating an exclusion
A practical method to analyze an Azure WAF block, isolate the rule involved, compare application evidence, and decide between a fix, a targeted exclusion, or a custom rule.
Read articleWAF and KQL: identify a false positive before creating an exclusion
A KQL analysis method to qualify an Azure WAF block, distinguish attack, noise and application false positive, then document the decision before any exclusion.
Read articleAzure WAF: read Application Gateway blocks with KQL without chasing every layer
Build useful KQL queries to identify requests blocked by Azure Web Application Firewall on Application Gateway, with action, ruleId, URI, client IP, hostname and time window.
Read article