An employee opens the HR agent and asks, "How much PTO do I accrue each month?" A few minutes later, someone else asks, "Where is the official code of ethics policy?" Those sound like the same problem. They are not. The first person needs a grounded answer they can understand. The second person needs a link to the right document quickly, without interpretation. If you design for one experience, the other one feels broken.
That is usually where knowledge-agent projects start to get messy. “Grounding” can sound like one switch you turn on, but in practice it is a spectrum: from zero-code classic search, to agentic retrieval over a knowledge base, to a forced-grounding agent that synthesizes answers when synthesis is required. The easier way to think about it is this: who is doing the retrieval work, and what does the user need back?
This post walks through five working retrieval patterns for an “Ask HR” agent built on Copilot Studio, Azure AI Search, and Foundry IQ. Each one is running code in the companion sample repo: foundry-copilot-hr-policy-knowledge. Each has a clear “use this when,” and the five patterns share the same reusable knowledge base so you can layer them on without re-indexing. By the end, you should have a decision tree you can reuse for your own knowledge source, whether that is HR policy, product docs, or support runbooks.
Scope: companion sample for learning and experimentation, not production-ready deployment. Review the Azure Well-Architected Framework for reliability, security, cost, and operational hardening before you ship.
The scenario: one index, many front doors
Here is the setup. The sample answers employee questions from a small corpus of internal HR policy documents: PTO accrual, hiring rules, code of ethics, blood-borne pathogen procedures, and dozens more. Underneath every pattern is one foundation: an Azure AI Search index named hr-policy-index, populated by an indexer and skillset that chunk and vectorize the documents.
Patterns A, C, and the Hosted Agent query that index directly. Patterns A2 and B add a Foundry IQ knowledge base named hr-knowledge-base on top of the same index for agentic retrieval. That layering is the part to pay attention to. The retrieval assets stay separate from the orchestration layer, so you can start with the simplest pattern, prove value quickly, and move to a more capable one later without re-indexing.
Two questions that decide everything
Before we get into the patterns, it helps to define the two retrieval terms I use throughout the rest of the post:
- Classic search, index-first retrieval: one hybrid (keyword + vector) query against an Azure AI Search index, ranked and returned. Fast and predictable.
- Agentic retrieval, the knowledge base plans multiple sub-queries from the user's question, runs them in parallel, re-ranks, and merges the results before the agent composes an answer. Higher quality on complex, multi-part questions.
If you want the fuller picture of how these two approaches map to retrieval-augmented generation, the Azure AI Search team's RAG and generative AI overview walks through the trade-offs and uses a similar HR/PTO example.
Once those terms are clear, the decision tree comes down to three practical questions:
Q1: Do users need an answer or are they really trying to find the right document?
If they just need the document, stay on the locator path. If they need the policy explained or summarized, move into the answer-synthesis path.
QL: Is the content in a citation-friendly knowledge base?
For example, SharePoint content or Azure AI Search content with a reliable blob_url. If yes, Copilot Studio can usually handle this with native citation cards in Pattern A. If not, use Pattern C with the dual tool /api/lookup path so the agent can return the exact document link.
Q2: Do you actually need an LLM agent in the middle?
If the answer is no, keep it simple: use classic search or agentic retrieval over the knowledge base. If the answer is yes, move into the agent path.
QK: For that non-agent path, is classic index search enough, or do you need agentic KB retrieval?
Classic search points to Pattern A. Agentic retrieval over the knowledge base points to Pattern A2.
Q3: If you need an agent, do you want Foundry to run the request loop, or do you need to self-host it?
If Foundry can manage the runtime, use Pattern B. If you need the request loop in your own container, use the Hosted Agent.
That is the decision tree in plain terms: Q1 decides whether this is a document-locator experience or an answer-synthesis experience. Q2 decides whether you need an LLM agent at all. Q3 is only about where the agent runs, either Foundry or your container. It does not change the front door; Copilot Studio can still be the user-facing experience.
How the sample repo is organized
The repo follows the same flow as the post. Start with docs/DataPipelineAndTesting.md to understand how the HR policy corpus is indexed, tested, and validated. Use docs/RetrievalPatterns.md as the decision model for choosing between classic search, agentic retrieval, forced grounding, and hosted runtime options. Then use the pattern-specific docs when you are ready to wire each path.
For Copilot Studio patterns, docs/CopilotStudioIntegration.md maps to Pattern A, while docs/CopilotStudioHybridExample.md maps to Pattern C and the dual-tool locator flow. For the more advanced agent paths, docs/FoundryAgentArchitecture.md covers Pattern B and the hosted agent architecture. docs/Distribution-M365-Teams.md shows how the agent can be distributed through Microsoft 365 and Teams once the retrieval pattern is working.
The rest of the post is that tree, one branch at a time.
Pattern A: Direct index (classic search, zero agent code)
Start here. Copilot Studio queries hr-policy-index directly through its built-in Knowledge action. No custom agent code runs in the answer path. The sample only owns the index, skillset, and indexing pipeline.
Populate the index (server-side indexer + skillset handles chunking and vectorization):
uv run python scripts/index_knowledge_base_integrated_vectorization.py # Builds hr-policy-index; a client-side alternative exists for dev/testWhat you get: very low latency in the sample, roughly 1-2 seconds, no LLM cost in the retrieval path, and native citation cards. When the source documents carry a blob_url or metadata_storage_path, Copilot Studio can render a click-through card straight to the document. For many "where is the policy?" questions, that may be enough.
The honest limitation: Pattern A is still classic search. It does not force synthesis. If Copilot Studio generates an answer from retrieved snippets, it may paraphrase a policy in a way that is close, but not precise enough. For HR policy, that matters. If exact wording matters, that is your sign to step up to Pattern B.
Pattern A2: Copilot Studio meets Foundry IQ (agentic retrieval, no prompt agent)
This is the pattern I would look at when you want better retrieval quality without taking on the overhead of a full prompt agent. In the Copilot Studio new agent experience preview, an agent connects directly to a Foundry IQ knowledge base through Microsoft IQ, with no Foundry prompt agent in between. You reuse the same hr-knowledge-base on top of the same hr-policy-index (one command: python -m src.agents.create_foundry_agent), but retrieval is now agentic: the knowledge base plans sub-queries, retrieves in parallel, reranks, and hands merged results to the agent.
Wiring it takes a few clicks in Copilot Studio (step-by-step on Microsoft Learn):
- Build → Microsoft IQ → Foundry IQ → Create new connection
- Choose Microsoft Entra ID Integrated authentication
- Select hr-knowledge-base
- Add to agent
A2 is worth the upgrade from A for two reasons. First, you get agentic-retrieval quality without having to build, deploy, or maintain a prompt agent. The knowledge base becomes the reusable asset you improve in Microsoft Foundry, not something you keep reworking inside each Copilot Studio agent. Second, when configured with Microsoft Entra ID Integrated authentication, retrieval can return ACL-trimmed results per user. Each person sees content based on their access.
Foundry IQ knowledge bases can also inherit enterprise-readiness controls such as customer-managed keys, network isolation, and Entra ID. A single knowledge base can also federate across multiple knowledge sources in parallel.
Use A2 when you want stronger hybrid retrieval quality without taking on the overhead of operating a full agent.
Pattern B: Foundry Agent Service with forced grounding
When answers need to be synthesized and grounded, publish a prompt agent to Microsoft Foundry with Foundry Agent Service. In the sample, the agent uses an MCPTool pointing at the knowledge-base endpoint, with tool_choice="required" so the model retrieves policy chunks before answering.
# src/agents/hr_policy_agent.py (excerpt) agent = PromptAgentDefinition( model=model_deployment_name, # e.g. gpt-5-mini instructions=HR_POLICY_INSTRUCTIONS, tools=[mcp_tool], # KB MCP endpoint tool_choice="required", # require retrieval before answering )Invoke it through the OpenAI client the project hands you:
client = project.get_openai_client() response = client.responses.create( input="How does PTO accrue for a new hire?", extra_body={"agent_reference": {"name": agent_name}}, )What you get: synthesized answers with grounding and inline [Policy XXXX - Title] citations, all from a single SDK call on a managed runtime. The trade-off: synthesis takes longer. In the sample, answers take roughly 10-14 seconds versus 1-2 seconds for classic search. For policy explanations, that extra time can be worth it because the user gets a composed, grounded answer instead of a list of snippets.
Pattern C: Dual-tool routing for deterministic document locators
Some questions do not need an essay; they just need the right URL, fast. Pattern C lets Copilot Studio route per turn:
- "Where is the PTO policy?" → POST /api/lookup, a deterministic endpoint with no LLM, roughly 1-2 seconds, returning the document URL verbatim in the answer body.
- "How many PTO hours do I accrue?" → hand off to Pattern A or B for a synthesized answer.
Reach for Pattern C when native citations are not enough. For example, use it when you need fast locator responses, the URL printed directly in the answer body, deterministic and auditable output, or support for a source that is not citation-friendly. The endpoint lives at src/backend/main.py:/api/lookup, with its contract in copilot/openapi-lookup-v2.json.
Hosted Agent: the same agent on your own runtime
If you need to own the request loop, custom authentication, side-car services, or infrastructure that stays inside your boundary, run the agent yourself. The Hosted Agent is the self-hosted version of the same idea: a container built on Microsoft Agent Framework with FoundryChatClient. It supports both classic and agentic retrieval through one environment variable:
|
RETRIEVAL_MODE |
Strategy |
Retrieval type |
|
tool (default) |
Custom @tool search_hr_policies (hybrid + semantic) |
Classic search |
|
context-semantic |
Built-in AzureAISearchContextProvider before each turn |
Classic search |
|
context-agentic |
AzureAISearchContextProvider over hr-knowledge-base |
Agentic retrieval |
The context-* modes use Agent Framework’s out-of-the-box RAG context provider. Retrieval runs automatically before each model call with standardized context and citation prompts, so the agent does not have to call a search tool explicitly. That gives the self-hosted path parity with the managed Foundry path across both retrieval types. Copilot Studio can still be the front door. Q3 in the decision tree is really about where the request loop runs, not who greets the user.
Choosing a pattern
|
Pattern |
Orchestrator |
Retrieval |
Latency (sample) |
Best for |
|
A |
Copilot Studio |
Classic |
~1-2 s |
Start here, native citations, no agent code |
|
A2 |
Copilot Studio → Foundry IQ |
Agentic |
~2-4 s |
Agentic quality, no agent to maintain |
|
B |
Foundry Agent Service |
Classic/agentic via MCP |
~10-14 s |
Forced-grounding synthesis in Foundry |
|
C |
Copilot Studio (router) |
None for lookup |
~1-2 s |
Deterministic, verbatim document locators |
|
Hosted |
Agent Framework container |
Classic + agentic |
~10-14 s |
Self-hosted runtime, custom auth |
A simple way to read the table: start at A, move to A2 when you want agentic retrieval without operating an agent, choose B when each answer needs to be synthesized and grounded in Foundry, add C for high-volume locator traffic, and pick the Hosted Agent when you need the runtime on your own infrastructure. These are not mutually exclusive. A mature agent often routes locator queries to C and content questions to A2 or B.
What's next?
- Try it: clone the sample and follow Steps 1-3 of the walkthrough to stand up Pattern A, provision hr-knowledge-base, connect Copilot Studio, and ask a question in minutes.
- Go agentic: wire the same knowledge base into the Copilot Studio new agent experience via Foundry IQ (Pattern A2) and compare answer quality side by side.
- Learn more: explore agentic retrieval in Azure AI Search, Foundry IQ, and Microsoft Agent Framework.
- Adapt it: swap the HR policy corpus for your own product docs, support runbooks, or internal knowledge source, then compare Pattern A, A2, and B against the same user questions.
- Use the repo-doc map: start with docs/RetrievalPatterns.md for the decision model, docs/CopilotStudioIntegration.md for Pattern A, docs/CopilotStudioHybridExample.md for Pattern C, docs/FoundryAgentArchitecture.md for Pattern B and Hosted Agent, and docs/DataPipelineAndTesting.md for ingestion and validation.
My recommendation: start simple, prove the index works, and move up the stack only when the use case needs it. Some questions need a trusted link. Others need a grounded explanation. A strong architecture supports both without forcing every request through the same path.
References
Copilot Studio + Foundry IQ
Foundry IQ / knowledge layer
Azure AI Search, retrieval
- Agentic retrieval overview
- RAG and generative AI in Azure AI Search
- Classic vs agentic search
- Create a knowledge base
- Create a knowledge source
- Hybrid search
- Semantic ranking
- Quickstart: agentic retrieval
- Tutorial: end-to-end agentic retrieval solution
Microsoft Foundry Agent Service (Pattern B)
Microsoft Agent Framework (Hosted Agent)
Governance
Related Microsoft Foundry blog posts













