AgentOps: diagnose MCP tool shadowing before a production action
A production runbook for detecting MCP tool collisions, proving which tool the agent actually selected and validating a canary catalog before any sensitive action.
Read article
Tag
microsoft-foundry
27 articles connected to this technical signal.
Azure AI Search: diagnose a partial indexer failure before resetting it
A production runbook for separating source, tracking, skillset and document failures in Azure AI Search before rerunning, targeting recovery or resetting the indexer.
Read articleAgentOps: contain sensitive data in agent traces before disabling observability
A production runbook for bounding a leak in agent traces, locating the unsafe field, canarying redaction and restoring useful observability.
Read articleAgentOps: validate a fallback model before enabling multi-model routing
A production runbook for proving that an agent fallback model preserves structured outputs, tool boundaries, refusals and traceability before routing live traffic.
Read articleAzure API Management AI gateway: diagnose token-limit 429s before raising the quota
A production runbook for separating APIM counters, policy scope, token estimation, backend throttling and client retries before expanding an AI token limit.
Read articleAgentOps: calibrate the evaluator before trusting an agent quality gate
A production runbook for separating agent regression from evaluator drift by freezing outputs, replaying an adjudicated anchor set, measuring disagreement and keeping promotion rollbackable.
Read articleMicrosoft Foundry: diagnose a content-filter spike before weakening guardrails
A production runbook for separating blocked prompts, filtered outputs, application errors and policy drift before changing Microsoft Foundry guardrails.
Read articleAzure API Management AI gateway: validate a backend circuit breaker before production failover
A production runbook for qualifying circuit breakers, Retry-After, Microsoft Foundry backend pools, canary traffic, observability and rollback before enabling AI API failover.
Read articleAgentOps: invalidate stale evidence before a production action
A production runbook for bounding evidence age across retrieval, telemetry and tool results before an AI agent proposes, executes or rolls back an operational action.
Read articleAgentOps: validate regional failover before rerouting a production agent
A production runbook for qualifying an AI agent regional failover across model deployment, state, retrieval, tools, identity, traces, canary traffic, validation and rollback.
Read articleAgentOps: revalidate a stale approval before resuming a production action
A production runbook for detecting and revalidating stale human approval across state drift, request fingerprints, expiry, traces, refusal tests and rollback before an AI agent resumes an action.
Read articleAgentOps: diagnose context-window saturation before raising token limits
A production runbook for reconstructing an AI agent's context, isolating history, retrieval and tool outputs, then validating compaction or rolling back.
Read articleMicrosoft Foundry Toolbox: validate a version before promoting it to agents
A production runbook to test a Toolbox version, its MCP tools, identities, approvals and traces before promotion, then return to the previous version without redeploying agents.
Read articleAgentOps: validate tool output before a production write
A production runbook for qualifying structured AI agent tool output with schema, sources, diff, idempotence, policy, traces, human validation and rollback before writing to production.
Read articleAgentOps: diagnose permission drift before widening an agent's rights
A production runbook for qualifying AI agent permission drift with the real identity, RBAC, tool scopes, traces, expected denials, human validation and rollback before widening access.
Read articleAgentOps: validate the Foundry automation handoff before a production action
A production runbook for qualifying the handoff from a Microsoft Foundry agent to Azure Automation, AWX, Azure DevOps or an MCP tool with contract, identity, traces, dry run, human validation and rollback.
Read articleAgentOps: diagnose a failed tool call before retrying a production action
A production runbook for qualifying a failed AI agent tool call with trace evidence, idempotence, identity, backend state, approvals, validation and rollback before retrying.
Read articleAgentOps: validate agent memory before production actions
A production runbook for qualifying AI agent memory with sources, traces, aging, permissions, evaluations, guardrails, human validation and rollback before it influences real actions.
Read articleAgentOps: validate an evaluation set before promoting an AI agent
A production runbook for qualifying an AI agent evaluation set with business cases, retrieval, tool calls, traces, thresholds, human validation and rollback before promotion.
Read articleMicrosoft Foundry: validate guardrails before exposing an agent to production
A production runbook for qualifying Microsoft Foundry agent guardrails with sources, refusals, tools, identity, traces, canary, human validation and rollback before user exposure.
Read articleAgentOps: diagnose prompt injection in retrieval before production actions
A production runbook for qualifying suspected prompt injection in an AI agent retrieval corpus with sources, traces, evaluation, tools, guardrails, validation and rollback.
Read articleAgentOps: validate an approval policy before production actions
A production runbook for qualifying an agent approval policy change with action scope, identity, traces, evaluation cases, guardrails, human validation and rollback.
Read articleAgentOps: diagnose Azure OpenAI rate limits before changing models
A production runbook for qualifying Azure OpenAI or Microsoft Foundry throttling with quota, deployment capacity, agent traces, retries, fallback, validation and rollback before changing models.
Read articleAgentOps: rotate an agent runtime identity before tool access fails
A production runbook for rotating or revoking an AI agent runtime identity with scoped permissions, dry-run tool calls, traces, approvals, validation and rollback before breaking production actions.
Read articleAgentOps: validate an agent tool before it can change production
A production runbook for qualifying a new AI agent tool with contract review, scoped identity, dry run, traces, approvals, evaluation cases and rollback before enabling real actions.
Read articleAgentOps: validate a retrieval index update before it changes production answers
A production runbook for qualifying a retrieval index update with source diff, metadata, chunking, evaluations, traces, human validation and rollback before changing an AI agent's answers.
Read articleMicrosoft Foundry: evaluate an agent before giving it a production action
A runbook for validating an AI agent before real action by separating sources, tools, identity, evaluation cases, human approvals, logs and rollback.
Read article