Azure Network Watcher: migrate NSG flow logs without losing network evidence

Azure NSG flow logs can no longer be created and retire on September 30, 2027. Leaving the migration to the final maintenance window turns a known platform change into an observability incident. Historical files continue to follow storage retention, but NSG flow log resources and their Traffic Analytics path are no longer a durable operating target.

The use case is an Azure hub-and-spoke platform where several NSG flow logs write to a storage account and a Log Analytics workspace. KQL queries, alerts and incident runbooks still depend on AzureNetworkAnalytics_CL. The objective is not merely to enable virtual network flow logs. The team must prove that the new scope captures useful traffic, move consumers to NTANetAnalytics, and retire the old path without a visibility gap or prolonged duplicate collection.

Freeze the observability contract

Start with the decisions the logs must support. A resource inventory alone does not prove that operators will still be able to explain a denied connection, unexpected egress or inter-VNet flow after cutover.

scope:
subscriptions: [prod-connectivity, prod-workloads]
regions: [westeurope]
virtual_networks: [vnet-hub-prod, vnet-app-prod]
evidence_to_preserve:
- allowed_and_denied_flows
- source_destination_protocol_port
- nsg_and_rule_when_available
- bytes_packets_and_flow_count
- traffic_analytics_queries
consumers:
- soc-network-workbook
- denied-flow-alert
- incident-network-pack
acceptance:
overlap_hours: 24
max_ingestion_delay_minutes: 70
required_queries_green: 4
rollback:
keep_nsg_flow_logs_enabled_until_acceptance: true
delete_new_flow_log_only_after_evidence_export: true

Set the accepted delay with Traffic Analytics processing in mind. Blobs are emitted in intervals and aggregated before they reach Log Analytics. An empty immediate query is not yet proof that collection failed.

Inventory flow logs and their consumers

List existing resources, targets, retention and Traffic Analytics settings. Then inventory queries, workbooks, alerts and exports that depend on the old table. Those consumers are usually the actual migration risk.

SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
az account set --subscription "$SUBSCRIPTION_ID"

az network watcher flow-log list --location westeurope --query "[].{name:name,enabled:enabled,target:targetResourceId,storage:storageId,analytics:flowAnalyticsConfiguration.networkWatcherFlowAnalyticsConfiguration.enabled,workspace:flowAnalyticsConfiguration.networkWatcherFlowAnalyticsConfiguration.workspaceResourceId,interval:flowAnalyticsConfiguration.networkWatcherFlowAnalyticsConfiguration.trafficAnalyticsInterval}" --output table

Group NSG flow logs by the VNet, subnet and NIC coverage they actually provide. Aggregating at VNet scope simplifies configuration and avoids duplicate records, but it can widen coverage compared with logging a small set of interfaces. Accept, cost and document that change before creating resources.

Use the Network Watcher migration script when configurations differ or logging covers only selected subnets and NICs. Azure Policy is a better fit when the target configuration is uniform and must apply to current and future VNets. In both cases, run analysis first and retain its report with the change record.

Prove prerequisites before parallel collection

The new resource needs a compatible storage destination, and Microsoft.Insights must be registered. Verify the identity creating the flow log as well as permissions required by Traffic Analytics on the workspace.

az provider show --namespace Microsoft.Insights --query "registrationState" --output tsv

az storage account show --ids "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-observability-prod/providers/Microsoft.Storage/storageAccounts/stflowprod" --query "{name:name,location:primaryLocation,httpsOnly:enableHttpsTrafficOnly,networkDefaultAction:networkRuleSet.defaultAction}" --output yaml

Do not broadly open the storage account to make the migration pass. If writes fail, separate authorization, storage network controls and flow log configuration. The correction should stay scoped and reversible.

Enable one canary virtual network flow log

Start with a representative VNet, using the same storage account and workspace as the existing path. Keep its NSG flow log enabled for a short comparison window.

RG_NETWORK="rg-network-prod"
RG_OBS="rg-observability-prod"
VNET="vnet-app-prod"
FLOW_LOG="fl-vnet-app-prod-weu"
STORAGE_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/$RG_OBS/providers/Microsoft.Storage/storageAccounts/stflowprod"
WORKSPACE_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/$RG_OBS/providers/Microsoft.OperationalInsights/workspaces/log-prod"

az network watcher flow-log create --location westeurope --resource-group "$RG_NETWORK" --name "$FLOW_LOG" --vnet "$VNET" --storage-account "$STORAGE_ID" --traffic-analytics true --workspace "$WORKSPACE_ID" --interval 10 --enabled true

Do not roll out broadly yet. Generate controlled probes: one allowed connection, one expected NSG deny, one intra-VNet flow and, where relevant, one inter-VNet flow. Record UTC timestamps, IP addresses, ports and expected outcomes.

Validate the new KQL schema

Traffic Analytics uses NTANetAnalytics for virtual network flow logs, replacing AzureNetworkAnalytics_CL used by NSG flow logs. Renaming the table is not enough. Several columns lose their type suffixes, and the collection scope changes.

let StartTime = ago(2h);
let CanarySources = dynamic(["10.42.3.17", "10.42.3.18"]);
NTANetAnalytics
| where TimeGenerated > StartTime
| where SubType == "FlowLog"
| where SrcIp in (CanarySources) or DestIp in (CanarySources)
| summarize
  Flows = sum(AllowedInFlows + DeniedInFlows + AllowedOutFlows + DeniedOutFlows),
  Bytes = sum(BytesSrcToDest + BytesDestToSrc),
  FirstSeen = min(FlowIntervalStartTime),
  LastSeen = max(FlowIntervalEndTime)
by SrcIp, DestIp, DestPort, L4Protocol, FlowStatus, AclRule
| order by LastSeen desc

Confirm exact field names in your workspace because instrumentation versions and local projections can vary. Start with NTANetAnalytics | getschema, then adapt the queries. Validation must answer four questions: do all probes appear, can the team distinguish allowed and denied traffic, are aggregate volumes plausible, and does ingestion meet the accepted delay?

During overlap, avoid a row-for-row comparison. The two solutions differ in scope and deduplication. Compare known canary flows and aggregate trends over a stable window.

Migrate consumers before disabling the source

Create an NTANetAnalytics version of every critical query, then point one canary workbook or alert at it. Maintain an explicit acceptance matrix.

Consumer                  Old source                 New source          Status
SOC network workbook       AzureNetworkAnalytics_CL    NTANetAnalytics     validated
Denied flow alert          AzureNetworkAnalytics_CL    NTANetAnalytics     shadow
Incident network pack      AzureNetworkAnalytics_CL    NTANetAnalytics     validated
Monthly traffic export     storage NSG path            storage VNet path   pending

Cutover blocked while any critical consumer is pending.

Run alert queries in shadow mode first: same window, same canary cases and no on-call notification. Compare missed events, noise and delay. Change the alert source separately from disabling old logs so each action remains attributable and reversible.

Decide cutover and prepare rollback

Cut over only when VNet flow log resources are healthy, blobs arrive, critical queries pass and wider coverage does not introduce unaccepted cost or data exposure. Disable NSG flow logs at that point, but do not delete them immediately.

decision: cutover
evidence:
vnet_flow_log_enabled: true
raw_blobs_present: true
traffic_analytics_delay_minutes: 34
canary_flows_found: 4/4
critical_consumers_validated: 3/3
storage_cost_reviewed: true
change:
disable_nsg_flow_logs: true
delete_nsg_flow_logs: false
validation_after_change:
- rerun four canary flows
- confirm NTANetAnalytics ingestion
- confirm workbook and shadow alert results
rollback_if:
- a canary flow is absent after the accepted delay
- a critical query cannot explain a known denial
- ingestion or storage authorization fails
rollback_action:
- re-enable the previous NSG flow logs
- keep the VNet flow log for diagnosis
- restore consumers to their previous queries

Delete old flow log resources only after an observation period and after retaining the migration evidence. Deleting a flow log does not delete existing records from the storage account. Those records continue to follow the configured lifecycle and retention policy.

Conclusion

Migrating NSG flow logs is more than creating another Network Watcher resource. It changes collection scope, the Traffic Analytics schema and every operational tool that turns a network flow into a decision.

A healthy outcome is testable: complete inventory, parallel canary collection, validated NTANetAnalytics queries, migrated consumers, disabled legacy logs and later deletion. If critical evidence is missing, re-enable the old path and correct the new one instead of opening an NSG rule to compensate for an observability failure.