Azure WAF: diagnose rate limiting before increasing the threshold

An Azure WAF rate limit can protect an application and break a legitimate journey in the same minute. Requests are blocked, users see 403 or 429 depending on the exposed layer, application teams ask to raise the threshold, and security teams worry about reopening a path for real abuse. Under pressure, the bad reflex is to double the threshold and watch the incident calm down. That can hide a bot, a retry storm, a poor counting key or an integration that concentrates all traffic behind one egress IP.

The use case is an application exposed through Azure Application Gateway WAF or Azure Front Door WAF with a rate-limiting custom rule. Since a deployment or a business campaign, part of the traffic is blocked. The runbook goal is not to find the perfect threshold during an incident. It is to decide whether the block is expected, whether the counting key is appropriate, whether the traffic is legitimate, whether a targeted exception is enough, or whether the rule should be rolled back.

Freeze the rate-limiting contract

Start by describing what the rule was meant to protect. A rate limit is not just a number per minute. It is a contract between an application path, a client population, a counting key, a time window and an action.

Observed rule
Surface: Application Gateway WAF or Azure Front Door WAF
Policy: waf-prod-public-api
Rule: rl-login-submit
Hostname: api.example.com
Path: /auth/login
Window: 1 minute
Threshold: 120 requests
Expected counting key: client IP, IP + header, or another supported signal
Action: Block or Log

Questions before change
Does the rule protect a sensitive endpoint or a normal journey?
Was the threshold based on real traffic or set defensively?
Does a gateway, proxy or NAT aggregate many users?
Does the block affect a few clients, a country, a tenant or all traffic?
Is there a previous policy version or a fallback to Detection mode?

Without this contract, the team argues about an abstract threshold. With it, the team can separate rule error, legitimate peak, abuse and application failure.

Measure the block before touching the threshold

The first evidence pack should show blocked volume, paths, clients, time distribution and the exact rule. The query below is a base to adapt to the WAF diagnostic fields available in your workspace.

let Window = 2h;
let Hostname = "api.example.com";
AzureDiagnostics
| where TimeGenerated > ago(Window)
| where Category == "ApplicationGatewayFirewallLog"
| extend hostname = tostring(host_s)
| extend uri = tostring(requestUri_s)
| extend action = tostring(action_s)
| extend ruleName = tostring(ruleName_s)
| extend ruleId = tostring(ruleId_s)
| extend clientIp = tostring(clientIp_s)
| extend transactionId = tostring(transactionId_g)
| where hostname == Hostname
| where action in ("Blocked", "Matched")
| where ruleName has "rate" or ruleId has "rate"
| summarize hits=count(), clients=dcount(clientIp), samples=make_set(uri, 8), firstSeen=min(TimeGenerated), lastSeen=max(TimeGenerated) by bin(TimeGenerated, 5m), ruleName, ruleId
| order by TimeGenerated asc

If the logs do not clearly show the rate-limiting rule, do not change the threshold yet. First prove that the block comes from WAF and not from an API gateway, APIM, the application, a CDN, bot management or an upstream component.

Identify who consumes the quota

A rate limit often fails because the counting key does not represent the real user. One source IP may hide a corporate NAT, mobile proxy, integration egress, test runner or serverless function. Conversely, a bot may spread traffic across many addresses.

let Window = 2h;
AzureDiagnostics
| where TimeGenerated > ago(Window)
| where Category == "ApplicationGatewayFirewallLog"
| extend uri = tostring(requestUri_s)
| extend clientIp = tostring(clientIp_s)
| extend userAgent = tostring(userAgent_s)
| extend ruleName = tostring(ruleName_s)
| where ruleName has "rate"
| summarize hits=count(), uris=make_set(uri, 5), agents=make_set(userAgent, 5), firstSeen=min(TimeGenerated), lastSeen=max(TimeGenerated) by clientIp
| order by hits desc
| take 30

The goal is not to identify a person. It is to understand whether the quota is consumed by a legitimate population, a noisy integration, a test, a retry incident or hostile traffic.

Compare blocked and accepted traffic

An overly strict rule is not visible only in blocks. It appears when blocked requests are compared with accepted requests on the same journey. If blocking starts after application latency or 5xx errors rise, the rate limit may be a consequence of a retry storm.

let Window = 2h;
let PathPrefix = "/auth/login";
AzureDiagnostics
| where TimeGenerated > ago(Window)
| where Category in ("ApplicationGatewayAccessLog", "ApplicationGatewayFirewallLog")
| extend uri = tostring(requestUri_s)
| extend action = tostring(action_s)
| extend status = toint(httpStatus_d)
| where uri startswith PathPrefix
| summarize total=count(), blocked=countif(action == "Blocked"), errors=countif(status >= 500), slowSamples=countif(timeTaken_d > 3) by bin(TimeGenerated, 5m)
| order by TimeGenerated asc

If errors or latency rise before blocks, raising the threshold may push more traffic into an already degraded backend. The better action may be an application rollback, client-side dampening, a cache, or a tighter temporary protection on one precise route.

Test the rule in a bounded mode

Before increasing the production threshold, prepare a bounded validation. The right test depends on your surface: Detection mode, preproduction replay, a more specific custom rule, an exception limited to the path, or a temporary observation rule.

Validation before change
Replay a sample of legitimate traffic with correlation ID
Confirm abusive requests still get blocked
Measure how many real clients sit behind the top IPs
Compare application p95/p99 before and after the change
Keep the change window short
Name the security and application decision owners

Acceptable change
Threshold adjusted on a precise path
More specific rule instead of a global threshold
Detection period limited in time
Temporary exception for an identified integration
Policy rollback ready and tested

A global threshold increase should remain the exception. If the issue comes from a coarse counting key, the real fix is to change the rule or entry-path design, not only the number.

Decide to raise, target or roll back

The decision should connect WAF evidence, application metrics and security risk. It must say what to do now and what will be checked after the change.

Raise the threshold
Legitimate traffic is proven
Backend is healthy during the window
Counting key is coherent
No abuse or enumeration signal
New value is justified by measurement
Post-change monitoring is active

Target the rule
A few IPs aggregate legitimate users
One endpoint or integration carries the peak
A global threshold would weaken the rest of the site
A host, path, method or header condition reduces risk

Block the change
Source of block is not proven
Retry storm or degraded backend
Hostile traffic is likely
No policy rollback
No application and security validation

Roll back
Backend errors rise after the change
Legitimate blocks persist
Protection bypass is observed
Security alerts or WAF metrics become inconsistent

Rollback must restore the previous policy, rule mode, priority and associated conditions. Returning only to the previous threshold is not enough if an exception or priority changed during the incident.

Conclusion

An operable WAF rate limit is not driven by intuition. It is driven by a clear contract, readable logs, understanding of the counting key, comparison with application health and a reversible decision.

The right outcome is not always to increase the threshold. Sometimes it is to target the rule, fix a noisy integration, address backend latency, move temporarily to observation, or roll back a policy that is too broad. The threshold should change only when the team can explain which traffic will be released, which risk remains blocked and how to go back.