AgentOps: validate the Foundry automation handoff before a production action

A Microsoft Foundry agent becomes operationally useful when it does more than summarize an incident. It reads sources, qualifies the symptom, prepares an action, then hands that action to automation: Azure Automation, AWX, Azure DevOps, a Logic App, an internal Function or a tool exposed through MCP. That is also when the risk changes. A poor summary is a diagnostic error. A poor handoff can start the wrong job in the wrong environment.

The use case is an operations assistant used by an on-call team. It can propose a runbook rerun, open a change request, prepare an application rollback or collect evidence. Before the team allows handoff to production automation, the runbook must decide whether the action can proceed, whether it should remain a draft, whether additional human validation is required, or whether the integration must be rolled back.

Define the handoff contract

The handoff should not be free text passed to a tool. It should be a contract: intent, environment, target, parameters, identity, execution mode, approval and expected rollback. Without that contract, the agent can produce a plausible instruction that is impossible to audit.

handoff:
agent: ops-assistant-prod
platform: Microsoft Foundry
target_tool: azure-automation-runbook
intent: collect_diagnostics
environment: production
mode: dry_run_then_approved_execution
approval: required_for_state_change

required_fields:
- incident_id
- source_documents
- target_resource
- bounded_action
- parameters
- runtime_identity
- expected_output
- rollback_or_cancel_path
- human_approver

reject_when:
- environment_missing
- target_resource_ambiguous
- action_changes_state_without_approval
- parameters_from_memory_without_source
- rollback_path_missing

This contract must be understood by the agent, by the called tool and by the human approver. It replaces vague autonomy with an action path the team can explain.

Separate diagnosis, proposal and execution

An agent can help diagnose without being allowed to execute. It can also prepare an action without starting it. Separating the stages is the first operational guardrail: read, propose, validate, execute, verify.

Handoff states
observed:
  The agent reads sources, traces and incident signals.

proposed:
  The agent proposes a bounded action with justification and parameters.

prepared:
  The tool receives a dry-run or draft request.

approved:
  A human confirms the environment, target and rollback.

executed:
  Automation runs with a dedicated identity and logs.

verified:
  The agent or team validates the expected effect with evidence.

Forbid
Direct observed -> executed transition
Parameters generated without a source
Production execution without approval
Reuse of an old remembered decision

This state model is deliberately simple. It prevents the team from confusing “the agent is probably right” with “the agent may act”.

Prove the identity that really acts

The handoff must identify both the agent identity and the automation identity. In many architectures, the agent does not modify anything directly. It calls a tool that uses its own managed identity, service connection, AWX credential or MCP runtime context. The control must focus on the identity that actually acts.

identity_chain:
conversation_user:
  role: requester
  can_approve: false

foundry_agent_identity:
  role: prepare_handoff
  allowed_actions:
    - read_sources
    - create_draft_action
    - call_dry_run_tool

automation_identity:
  role: execute_bounded_action
  allowed_scopes:
    - resource_group: rg-prod-observability
    - runbook: collect-app-diagnostics
  forbidden_scopes:
    - subscription_owner
    - wildcard_resource_group
    - firewall_policy_write

human_approver:
  role: approve_state_change
  evidence_required:
    - source_trace
    - dry_run_output
    - rollback_path

If the execution identity has more permission than the handoff contract allows, the issue is not the prompt. It is an operational architecture gap.

Test with dry run before action

A useful dry run does more than say “valid”. It returns what would be targeted, which commands would be called, which parameters would be used and why the action is allowed. The agent should include that output in the approval request.

{
"handoffId": "inc-7421-diag-001",
"mode": "dry-run",
"target": {
  "environment": "production",
  "resource": "app-api-prod",
  "scope": "single service"
},
"action": "collect_diagnostics",
"wouldRun": [
  "query application errors for 30 minutes",
  "export dependency failures",
  "attach summary to incident ticket"
],
"willNotRun": [
  "restart service",
  "change firewall rule",
  "modify RBAC"
],
"approvalRequired": false,
"rollback": "delete generated evidence package if wrong incident scope"
}

For a state-changing action, the dry run should instead return approvalRequired: true. That difference should come from the contract, not from an improvised agent decision.

Trace source, decision and tool

The trace must let the team rebuild the full chain: user request, consulted sources, useful reasoning, selected tool, parameters, approval, execution and result. Without that chain, the team will not know whether an incident came from a stale source, wrong parameter mapping, an over-permissive tool or a rushed approval.

let IncidentId = "INC-7421";
AgentHandoffEvents
| where TimeGenerated > ago(24h)
| where IncidentId == IncidentId
| project TimeGenerated,
        HandoffId,
        AgentName,
        ToolName,
        Intent,
        Environment,
        TargetResource,
        Mode,
        ApprovalState,
        RuntimeIdentity,
        SourceCount,
        TraceId
| order by TimeGenerated asc

Adapt table names to your observability setup. The important point is to search for a handoff object, not only for a conversation.

Block ambiguous parameters

The most common risk is not a radically dangerous tool. It is an overbroad parameter: prod instead of a service, a whole resource group instead of a resource, a default branch instead of a commit, a complete AWX inventory instead of a targeted limit.

Parameters to reject
environment: missing, all, any
target: wildcard, subscription, tenant, all-hosts
action: restart, delete, rotate, open-firewall without approval
duration: permanent for a temporary exception
source: agent memory without cited document
branch: latest without approved commit or tag

Acceptable parameters
environment: production with approver
target: single resource or explicit list
action: read-only collection or verification
duration: bounded window
source: approved document with version or date
branch: validated commit, tag or release

These rules should live near the tool, not only in the prompt. The tool must reject an overbroad request even if the agent phrases it nicely.

Decide execute, draft or rollback

The final decision must be explicit. The handoff can execute if the contract is complete, the dry run is coherent, the identity is bounded, sources are cited and rollback is available. It should remain a draft if the action is plausible but lacks approval. It should be rejected if the target or parameters are ambiguous. It should be rolled back if the handoff already created an action with the wrong scope.

Execute
Complete contract
Readable dry run
Bounded execution identity
Approved sources cited
Human validation present for state change
Post-action verification defined

Keep as draft
Plausible diagnosis
Incomplete parameters
Approver missing
Change window not confirmed

Reject
Ambiguous target
Overbroad action
Over-permissive identity
Unapproved source
Missing rollback

Rollback
Action prepared for wrong environment
Ticket or job created with wrong scope
Approval attached to wrong incident
Trace insufficient to audit execution

Rollback can be simple: disable the tool, return to mandatory dry run, cancel a prepared job, close a change request or restore the previous MCP contract version.

Validate after enablement

After enablement, do not measure only successful actions. Measure rejections, drafts, approvals, dry runs and rollbacks. A good handoff should produce understandable refusals when context is missing.

Post-enable validation
Read-only handoffs pass without unnecessary approval
State changes require human validation
Rejections explain the missing or dangerous field
Logs link source, tool, identity and action
Actions stay bounded to the requested environment
Rollback can disable the tool without losing traces

Alert signal
Too many actions in executed mode without dry run
Parameters corrected manually after the fact
Approvals granted outside the incident
Tool cannot explain its refusal
Execution identity shared with other automations

A reliable agent is not the one that triggers the most automation. It is the one that hands off fewer actions with enough evidence for the team to accept or reject them quickly.

Conclusion

The handoff between a Microsoft Foundry agent and production automation should be treated as an operations boundary. It needs a contract, bounded identity, dry run, human approval for state changes, traces and rollback.

The right decision is not always to execute. It may be to keep the action as a draft, reduce the MCP tool, return to mandatory dry run or temporarily disable the connector. The goal is straightforward: let the agent speed up operations without allowing a plausible suggestion to become an opaque production action.