AgentOps: diagnose MCP tool shadowing before a production action
A production runbook for detecting MCP tool collisions, proving which tool the agent actually selected and validating a canary catalog before any sensitive action.
Read article
Tag
guardrails
53 articles connected to this technical signal.
AgentOps: contain sensitive data in agent traces before disabling observability
A production runbook for bounding a leak in agent traces, locating the unsafe field, canarying redaction and restoring useful observability.
Read articleAgentOps: validate a fallback model before enabling multi-model routing
A production runbook for proving that an agent fallback model preserves structured outputs, tool boundaries, refusals and traceability before routing live traffic.
Read articleAgentOps: neutralize indirect prompt injection in MCP tool output before a production action
A production runbook for isolating hostile instructions carried by MCP tool output, preserving provenance, enforcing policy outside the model, and validating or rolling back write access.
Read articleAzure API Management AI gateway: diagnose token-limit 429s before raising the quota
A production runbook for separating APIM counters, policy scope, token estimation, backend throttling and client retries before expanding an AI token limit.
Read articleAgentOps: calibrate the evaluator before trusting an agent quality gate
A production runbook for separating agent regression from evaluator drift by freezing outputs, replaying an adjudicated anchor set, measuring disagreement and keeping promotion rollbackable.
Read articleAzure Container Registry: migrate Docker Content Trust to Notation without opening the delivery chain
A production runbook for inventorying Docker Content Trust, dual-signing with Notation, validating digests, and cutting over CI/CD gates with an explicit rollback.
Read articleAzure Resource Graph: prove inventory completeness before automated remediation
A production runbook for proving Azure Resource Graph scope, identity, pagination and result completeness before an inventory drives automated changes.
Read articleMicrosoft Foundry: diagnose a content-filter spike before weakening guardrails
A production runbook for separating blocked prompts, filtered outputs, application errors and policy drift before changing Microsoft Foundry guardrails.
Read articleAzure API Management AI gateway: validate a backend circuit breaker before production failover
A production runbook for qualifying circuit breakers, Retry-After, Microsoft Foundry backend pools, canary traffic, observability and rollback before enabling AI API failover.
Read articleAgentOps: prevent MCP token forwarding before production actions
A production runbook for proving token audience, delegation, scopes and runtime identity across an MCP tool chain before restoring state-changing actions.
Read articleMCP Roots is deprecated: migrate file scope without expanding access
A production runbook for replacing MCP Roots with tool parameters, resource URIs or server configuration while preserving path controls, canary evidence and rollback.
Read articleGitHub Actions: validate a reusable workflow before trusting it with an Azure deployment
A production runbook for qualifying a reusable GitHub Actions workflow through permissions, secrets, OIDC, environments, traces, canary validation and rollback before an Azure deployment.
Read articleAgentOps: invalidate stale evidence before a production action
A production runbook for bounding evidence age across retrieval, telemetry and tool results before an AI agent proposes, executes or rolls back an operational action.
Read articleAgentOps: validate regional failover before rerouting a production agent
A production runbook for qualifying an AI agent regional failover across model deployment, state, retrieval, tools, identity, traces, canary traffic, validation and rollback.
Read articleAgentOps: revalidate a stale approval before resuming a production action
A production runbook for detecting and revalidating stale human approval across state drift, request fingerprints, expiry, traces, refusal tests and rollback before an AI agent resumes an action.
Read articleMicrosoft Sentinel: validate a remediation playbook before production action
A production runbook for bounding a Microsoft Sentinel playbook across trigger, entities, identity, dry run, approval, traces, canary and rollback before any remediation action.
Read articleAgentOps: validate session and memory isolation before production
A production runbook for detecting context leaks across AI agent sessions, separating memory, caches, retrieval, tools and identity, then enabling or rolling back persistence without losing audit traces.
Read articleAgentOps: diagnose context-window saturation before raising token limits
A production runbook for reconstructing an AI agent's context, isolating history, retrieval and tool outputs, then validating compaction or rolling back.
Read articleAgentOps: detect evaluation set contamination before promoting an agent
A production runbook for proving that an AgentOps score is not inflated by leakage across evaluations, prompts, retrieval or tuning data before promoting an AI agent.
Read articleAzure ACR and AKS: verify an image signature before production promotion
A production runbook for pinning an ACR digest, verifying its Notation signature, scoping trust policy, promoting by digest, and rolling back without disabling provenance controls.
Read articleMicrosoft Foundry Toolbox: validate a version before promoting it to agents
A production runbook to test a Toolbox version, its MCP tools, identities, approvals and traces before promotion, then return to the previous version without redeploying agents.
Read articleAgentOps: stop a multi-agent handoff loop before actions multiply
A production runbook for detecting an agent delegation loop, containing side effects, rebuilding traces, and deciding controlled recovery or a single-agent rollback.
Read articleAgentOps: shadow a prompt or model change before production cutover
A production runbook for comparing a candidate agent with the active runtime on the same requests without duplicating actions, then deciding promotion, canary or rollback.
Read articleAgentOps: contain an AI agent consumption runaway before shutting down production
A production runbook for attributing token and tool-call growth, isolating amplification, enforcing an execution budget, and validating or rolling back an AI agent release.
Read articleAgentOps: bound MCP tool timeouts, retries and circuit breakers before production
A production runbook for defining latency budgets, idempotency, retries, circuit breakers, traces and rollback for an MCP tool called by an AI agent.
Read articleAgentOps: validate tool output before a production write
A production runbook for qualifying structured AI agent tool output with schema, sources, diff, idempotence, policy, traces, human validation and rollback before writing to production.
Read articleAzure Deployment Stacks: validate unmanage and deny settings before an update
A production runbook for reviewing managed resources, unmanage behavior, deny settings, excluded identities, validation and rollback before updating an Azure Deployment Stack.
Read articleAgentOps: diagnose permission drift before widening an agent's rights
A production runbook for qualifying AI agent permission drift with the real identity, RBAC, tool scopes, traces, expected denials, human validation and rollback before widening access.
Read articleAgentOps: validate the Foundry automation handoff before a production action
A production runbook for qualifying the handoff from a Microsoft Foundry agent to Azure Automation, AWX, Azure DevOps or an MCP tool with contract, identity, traces, dry run, human validation and rollback.
Read articleAgentOps: diagnose MCP server drift before a production action
A production runbook for qualifying MCP server drift with tool manifests, schemas, identity, secrets, network path, traces, evaluations, validation and rollback before reauthorizing an AI agent.
Read articleAgentOps: scope an internal MCP server before exposing it to an agent
A production runbook for qualifying an internal MCP server with tool inventory, scopes, identities, secrets, audit, dry runs, evaluations, human validation and rollback before agent access.
Read articleAgentOps: diagnose a failed tool call before retrying a production action
A production runbook for qualifying a failed AI agent tool call with trace evidence, idempotence, identity, backend state, approvals, validation and rollback before retrying.
Read articleAgentOps: validate agent memory before production actions
A production runbook for qualifying AI agent memory with sources, traces, aging, permissions, evaluations, guardrails, human validation and rollback before it influences real actions.
Read articleAgentOps: validate an evaluation set before promoting an AI agent
A production runbook for qualifying an AI agent evaluation set with business cases, retrieval, tool calls, traces, thresholds, human validation and rollback before promotion.
Read articleMicrosoft Foundry: validate guardrails before exposing an agent to production
A production runbook for qualifying Microsoft Foundry agent guardrails with sources, refusals, tools, identity, traces, canary, human validation and rollback before user exposure.
Read articleAgentOps: diagnose prompt injection in retrieval before production actions
A production runbook for qualifying suspected prompt injection in an AI agent retrieval corpus with sources, traces, evaluation, tools, guardrails, validation and rollback.
Read articleAgentOps: validate an approval policy before production actions
A production runbook for qualifying an agent approval policy change with action scope, identity, traces, evaluation cases, guardrails, human validation and rollback.
Read articleAgentOps: diagnose Azure OpenAI rate limits before changing models
A production runbook for qualifying Azure OpenAI or Microsoft Foundry throttling with quota, deployment capacity, agent traces, retries, fallback, validation and rollback before changing models.
Read articleAgentOps: rotate an agent runtime identity before tool access fails
A production runbook for rotating or revoking an AI agent runtime identity with scoped permissions, dry-run tool calls, traces, approvals, validation and rollback before breaking production actions.
Read articleAzure Automation: validate a scheduled runbook before it changes production
A production runbook for qualifying a scheduled Azure Automation job with trigger, managed identity, parameters, dry-run, logs, validation and rollback before allowing real action.
Read articleAgentOps: validate an agent tool before it can change production
A production runbook for qualifying a new AI agent tool with contract review, scoped identity, dry run, traces, approvals, evaluation cases and rollback before enabling real actions.
Read articleAgentOps: validate a retrieval index update before it changes production answers
A production runbook for qualifying a retrieval index update with source diff, metadata, chunking, evaluations, traces, human validation and rollback before changing an AI agent's answers.
Read articleAgentOps: diagnose missing agent traces before restoring a production action
A production runbook for qualifying incomplete AI agent traces with conversation events, sources, tool calls, identities, approvals, evaluations and rollback before restoring an action.
Read articleAWX: validate dynamic inventory before a production job
A production runbook for qualifying an AWX dynamic inventory change with source, host diff, variables, identities, limits, validation and rollback before execution.
Read articleAzure DevOps MCP: scope an agent before letting it act on the project
A production runbook for exposing Azure DevOps to an AI agent through MCP with project scope, permissions, allowed actions, evidence, logs, human validation and rollback.
Read articleAWX: rerun a failed job without replaying a partial action
A production runbook for qualifying a failed AWX job with Ansible events, changed tasks, affected hosts, variables, limited rerun, validation and rollback.
Read articleAWX snippet: capture failed job evidence before rerun
A short sequence to collect status, variables, affected host and error events before rerunning an AWX job.
Read articleAgentOps: diagnose agent contract drift before redeployment
A production runbook for qualifying AI agent contract drift with prompts, tool manifests, sources, evaluations, traces, human validation and rollback.
Read articleAgentOps: diagnose an AI agent action before rollback
A production runbook for qualifying an AI agent action with traces, sources, tools, identity, KQL, human validation and rollback without disabling the whole assistant.
Read articleMicrosoft Foundry: evaluate an agent before giving it a production action
A runbook for validating an AI agent before real action by separating sources, tools, identity, evaluation cases, human approvals, logs and rollback.
Read articleAgentOps: expose MCP tools without losing control of production actions
A production runbook for exposing MCP tools to an AI agent while keeping action scope, approvals, identities, logs, evaluation and rollback under control.
Read articleAgentOps: diagnose an AI agent that calls the wrong tool
A production runbook for qualifying an AI agent that selects the wrong tool, acts without evidence or hides an action behind a plausible answer.
Read article