When an incident fires at 3 AM, every second the on-call engineer spends piecing together alerts, logs, and metrics is a second not spent fixing the problem. What if an AI system could ingest the raw incident signals and hand you a structured triage, a Slack update, a stakeholder brief, and a draft post-incident report, all in under 10 seconds?

That’s exactly what On-Call Copilot does. In this post, we’ll walk through how we built it using the Microsoft Agent Framework, deployed it as a Foundry Hosted Agent, and discuss the key design decisions that make multi-agent orchestration practical for production workloads.

The full source code is open-source on GitHub. You can deploy your own instance with a single azd up.

Why Multi-Agent? The Problem with Single-Prompt Triage

Early AI incident assistants used a single large prompt: “Here is the incident. Give me root causes, actions, a Slack message, and a post-incident report.” This approach has two fundamental problems:

1. Context overload. A real incident may have 800 lines of logs, 10 alert lines, and dense metrics. Asking one model to process everything and produce four distinct output formats in a single turn pushes token limits and degrades quality.
2. Conflicting concerns. Triage reasoning and communication drafting are cognitively different tasks. A model optimised for structured JSON analysis often produces stilted Slack messages—and vice versa.

The fix is specialisation: decompose the task into focused agents, give each agent a narrow instruction set, and run them in parallel. This is the core pattern that the Microsoft Agent Framework makes easy.

Architecture: Four Agents Running Concurrently

On-Call Copilot is deployed as a Foundry Hosted Agent—a containerised Python service running on Microsoft Foundry’s managed infrastructure. The core orchestrator uses ConcurrentBuilder from the Microsoft Agent Framework SDK to run four specialist agents in parallel via asyncio.gather().

All four agents The solution share a single Azure OpenAI Model Router deployment. Rather than hardcoding gpt-4o or gpt-4o-mini, Model Router analyses request complexity and routes automatically. A simple triage prompt costs less; a long post-incident synthesis uses a more capable model. One deployment name, zero model-selection code.

Meet the Four Agents

The Code: Building the Orchestrator

The entry point is remarkably concise. ConcurrentBuilder handles all the async wiring—you just declare the agents and let the framework handle parallelism, error propagation, and response merging.

Agent Instructions: Prompts as Configuration

Each agent receives a tightly scoped system prompt that defines its output schema and guardrails. Here’s the Triage Agent—the most complex of the four:

The Comms Agent follows the same pattern but targets a different audience:

The Incident Envelope: What Goes In

The agent accepts a single JSON envelope. It can come from a monitoring alert webhook, a PagerDuty payload, or a manual CLI invocation:

Declaring the Hosted Agent

The agent is registered with Microsoft Foundry via a declarative agent.yaml file. This tells Foundry how to discover and route requests to the container:

The protocols: [responses] declaration exposes the agent via the Foundry Responses API on port 8088. Clients can invoke it with a standard HTTP POST no custom API needed.

Invoking the Agent

Once deployed, you can invoke the agent with the project’s built-in scripts or directly via curl:

The Browser UI

The project includes a zero-dependency browser UI built with plain HTML, CSS, and vanilla JavaScript—no React, no bundler. A Python http.server backend proxies requests to the Foundry endpoint.

Agent Output Panels

Performance: Parallel Execution Matters

asyncio.gather() running four independent agents cuts total latency by 3–4× compared to sequential execution. For a SEV1 at 3 AM, that’s the difference between a 10-second AI-powered head start and a 40-second wait.

Five Key Design Decisions

1. Parallel over sequential Each agent is independent and processes the full incident payload in isolation. ConcurrentBuilder with asyncio.gather() is the right primitive—no inter-agent dependencies, no shared state.
2. JSON-only agent instructions Every agent returns only valid JSON with a defined schema. The orchestrator merges fragments with merged.update(agent_output). No parsing, no extraction, no post-processing.
3. No hardcoded model names AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=model-router is the only model reference. Model Router selects the best model at runtime based on prompt complexity. When new models ship, the agent gets better for free.
4. DefaultAzureCredential everywhere No API keys. No token management code. Managed identity in production, az login in development. Same code, both environments.
5. Instructions as configuration Each agent’s system prompt is a plain Python string. Behaviour changes are text edits, not code logic. A non-developer can refine prompts and redeploy.

Guardrails: Built into the Prompts

The agent instructions include explicit guardrails that don’t require external filtering:

• No hallucination: When data is insufficient, the agent sets confidence: 0 and populates missing_information rather than inventing facts.
• Secret redaction: Each agent is instructed to redact credential-like patterns as [REDACTED] in its output.
• Mark unknowns: Undeterminable fields use the literal string "UNKNOWN" rather than plausible-sounding guesses.
• Diagnostic suggestions: When signal is sparse, immediate_actions includes diagnostic steps that gather missing information before prescribing a fix.

Model Router: Automatic Model Selection

One of the most powerful aspects of this architecture is Model Router. Instead of choosing between gpt-4o, gpt-4o-mini, or o3-mini per agent, you deploy a single model-router endpoint. Model Router analyses each request’s complexity and routes it to the most cost-effective model that can handle it.

This means you get optimal cost-performance without writing any model-selection logic. A simple Summary Agent prompt may route to gpt-4o-mini, while a complex Triage Agent prompt with 800 lines of logs routes to gpt-4o all automatically.

Deployment: One Command

The repo includes both azure.yaml and agent.yaml, so deployment is a single command:

# Deploy everything: infra + container + Model Router + Hosted Agent

This provisions the Foundry project resources, builds the Docker image, pushes to Azure Container Registry, deploys a Model Router instance, and creates the Hosted Agent. For more control, you can use the SDK deploy script:

# Build and push (must be linux/amd64)

# Create the hosted agent

Getting Started

Extending: Add Your Own Agent

Adding a fifth agent is straightforward. Follow this pattern:

1. Create app/agents/<name>.py with a *_INSTRUCTIONS constant following the existing pattern.
2. Add the agent’s output keys to app/schemas.py.
3. Register it in main.py:

Ideas for extensions: a ticket auto-creation agent that creates Jira or Azure DevOps items from the PIR output, a webhook adapter agent that normalises PagerDuty or Datadog payloads, or a human-in-the-loop agent that surfaces missing_information as an interactive form.

Key Takeaways for AI Engineers

• Microsoft Agent Framework gives you ConcurrentBuilder for parallel execution and AzureOpenAIChatClient for Azure-native auth—you write the prompts, the framework handles the plumbing.
• Foundry Hosted Agents let you deploy containerised agents with managed infrastructure, automatic scaling, and built-in telemetry. No Kubernetes, no custom API gateway.
• Model Router eliminates the model selection problem. One deployment name handles all scenarios with optimal cost-performance tradeoffs.
• Prompt-as-config means your agents are iterable by anyone who can edit text. The feedback loop from “this output could be better” to “deployed improvement” is minutes, not sprints.

Resources