Engineering teams across India and global tech hubs have built Retrieval-Augmented Generation into enterprise backends. Pairing Large Language Models with vector databases grounded AI responses in proprietary docs without costly fine-tuning. Yet, once systems moved beyond basic FAQ lookups, traditional vector search struggled with the bigger picture.

When enterprise clients ask AI assistants to trace dependencies across documents or summarize risks, vector search falls flat. This challenge led to Graph RAG, pairing Knowledge Graphs with semantic search. In this hands-on guide, we compare RAG vs. Graph RAG, analyze lab benchmarks, and explore when you need a graph database.

Table of Contents

• The Semantic Ceiling: Why Vector RAG Struggles with Complex Reasoning
• What Is Graph RAG? Deconstructing Knowledge Graphs in Generative AI
• Lab Benchmarks: 1,200 Enterprise Documents Tested Head-to-Head
• Architecture & Retrieval Mechanics: Local, Global, and DRIFT Search
• Comprehensive Comparison: Traditional RAG vs Graph RAG
• Strategic Decision Matrix: When to Choose Vector RAG vs Graph RAG
• The 2026 Golden Standard: Building Hybrid GraphRAG Pipelines
• Agentic AI & Model Context Protocol (MCP) Integration
• Frequently Asked Questions (FAQ)
• End Note

End Note

The evolution from traditional Vector RAG to Graph RAG reflects the maturation of enterprise AI. While vector embeddings opened the door to semantic discovery, knowledge graphs provide the contextual scaffolding required for complex reasoning, multi-hop discovery, and corpus-wide synthesis.

 

As indexing costs decrease and hybrid architectures mature, pairing graph topology with vector search is becoming the gold standard for reliable AI systems. Explore more deep architectural analyses, engineering guides, and machine learning tutorials across our dedicated Artificial Intelligence and Tech Guide hubs.

 

Claude Haiku 5.5 is now available in Microsoft Foundry. The fastest and most efficient model in the Claude 5.5 family, it is built for high-volume, cost-sensitive agentic work, from live customer support to focused subtasks within larger agent systems. It also costs around 75% less than Claude Haiku 4.5 for most tasks. 

As teams scale AI applications, the speed and cost of each repeated task matter. Haiku 5.5 gives developers a model built for that work, helping them keep experiences responsive and delegate routine execution so more capable models can focus on complex reasoning. 

A fast subagent for focused work 

Claude Haiku 5.5 is Anthropic’s most capable Haiku model yet, with significant improvements over Haiku 4.5 across coding, tool use, computer use, and agents. Its role in the Claude family is focused execution: handling well-defined tasks quickly and efficiently as part of a broader workflow. 

For coding agents, a model such as Claude Opus 5.5 can plan the work and delegate specific subtasks to Haiku 5.5. These might include routing requests, summarizing context, applying small, targeted changes across many files, or checking a proposed sequence of steps. This division of work makes it practical to run more subagents in parallel while reserving deeper reasoning for the hardest problems. 

Responsive experiences and everyday automation 

Haiku 5.5 is built for interactive experiences where users expect quick answers, including voice agents, live support, and in-app assistants. It can handle straightforward customer questions and quick knowledge-base lookups as part of these applications. 

Behind the scenes, it supports the repeatable work that keeps business processes moving: extracting key fields from small-to-medium documents, classifying incoming content, generating summaries, and flagging items that need a closer review. Its speed and efficiency make it well suited to processing these tasks at high volume. 

For browser and desktop automation, Haiku 5.5 can act as a computer-use subagent for repetitive tasks such as filling forms, entering data, and moving information between applications. Support for high-resolution images also enables work with screenshots, charts, and documents. 

More control over cost and intelligence 

Haiku 5.5 is the first Haiku model with effort controls, giving teams a way to tune the balance between cost and intelligence for each task. Developers can use these controls to match the model’s effort to the demands of a workflow, with a focus on the quality and economics their application needs. 

Get started in Microsoft Foundry 

Explore Claude Haiku 5.5 in Microsoft Foundry and evaluate it on the focused tasks your application performs most often. For teams building agent systems, it adds a fast, efficient option for the repeatable work that needs to scale. Try it today claude-haiku-4-5 | Model Catalog | Microsoft Foundry

In Part 1, we explored the intelligence side of the Microsoft AI and Agent Platform: how agents understand work, reason over trusted context, use models and tools, and operate across experiences such as Microsoft 365 Copilot, Copilot Studio, Microsoft Foundry, Dynamics 365, GitHub, and custom applications.

But intelligence is only half the equation.

As agents move from answering questions to taking action, the security model changes. Agents can retrieve sensitive data, call APIs, invoke tools, trigger workflows, collaborate with other agents, remember context, and act with delegated or assigned authority. That creates a new enterprise requirement: agents must be not only useful, but also governable, observable, contained, policy-driven, auditable, and aligned with organizational risk boundaries.

The core question for enterprise AI is no longer only, “Can the agent reason?” It is also, “Can we trust what it can access, what it can do, how it is governed, and whether we can prove what happened?”

That is the focus of Part 2: Trust—showing where security, governance, assurance, isolation, posture management, and security operations apply across the agent lifecycle.

The mental model: The left side presents the seven platform layers, while the foundation establishes security across the architecture. On the right, scoped control rails apply targeted governance through Microsoft commercial controls and open-source specifications, with each control operating within its defined scope.

Trust is not a feature. It is a platform property.

Many early AI security conversations focused narrowly on prompts: prompt injection, jailbreaks, harmful content, and model outputs. Those risks remain important, but agentic systems broaden the attack surface.

An enterprise agent is not just a model endpoint. It is a system composed of:

• User and application interaction channels.
• Foundation and custom models.
• Agent orchestration and reasoning.
• MCP servers, APIs, tools, skills, and connectors.
• RAG pipelines and enterprise knowledge sources.
• Short-term and long-term memory.
• Identity and access controls.
• Runtime environments, containers, cloud PCs, and networks.
• Monitoring, audit, compliance, and security operations.

Trust must therefore be applied across every layer. A secured agent platform needs controls before the model call, during reasoning, before and after tool execution, at data retrieval time, during output generation, and throughout runtime operation.

That is why the Microsoft Security for AI Platform is best understood as an end-to-end security architecture for the AI application and agent lifecycle. The diagram follows the path from interaction channels, through the agent and grounding layers, into build and runtime environments, and finally into security operations.

The new security boundary: from applications to agents

Traditional application security assumes relatively deterministic behavior: users click buttons, applications call APIs, authorization checks enforce access, and logs record the result. Agentic systems introduce more dynamic behavior.

An agent may interpret a goal, decompose it into tasks, choose tools, retrieve context, call APIs, invoke another agent, generate code, operate a browser, or take action through a workflow. That flexibility is where business value comes from, but it also means controls must cover more than static application paths.

A secure AI platform must answer several questions:

Microsoft’s approach is to apply a consistent trust foundation across the same layers that deliver intelligence: interaction channels, AI applications and agents, data and context, build platforms, runtime infrastructure, and security operations.

1. Secure the AI interaction channels

Agents enter the enterprise through many channels: Microsoft 365 Copilot, Teams, SharePoint, Office, Dynamics 365, custom portals, APIs, MCP endpoints, web experiences, and developer tools. Each channel has a different risk profile.

Microsoft 365 Copilot and business application experiences inherit enterprise identity, compliance, and data protection controls from Microsoft 365, Microsoft Entra, Microsoft Purview, and Microsoft Defender. Custom apps, web apps, and API-driven experiences require the same rigor: authentication, authorization, network controls, input validation, logging, monitoring, and response.

For developer tools, GitHub Advanced Security, Microsoft Defender for DevOps, Microsoft Defender for Cloud, and Microsoft Security Copilot help protect the software supply chain behind AI applications. Agent systems often depend on code, prompts, infrastructure-as-code, container images, packages, actions, connectors, and deployment pipelines. Those assets need the same scanning, secret detection, dependency governance, and posture management expected of any production system.

For APIs and MCP servers, Azure API Management, Microsoft Defender for APIs, Microsoft Entra, Conditional Access, Private Link, and network controls become especially important. Agent tool use should be explicit, least privileged, monitored, and policy governed.

Microsoft Defender for Cloud Apps also plays an important role across interaction channels by helping organizations discover and govern SaaS usage, manage OAuth app risk, apply session controls, and understand shadow AI exposure across the enterprise.

2. Secure the AI application and agent layer

The AI application and agent layer is where models, agents, MCP, skills, hooks, RAG, memory, and orchestration come together. This layer sets agent security apart because agents do more than process requests, they reason, make decisions, and take action.

Agent harness and the agent loop

An agent harness is the runtime scaffolding that turns model reasoning into a stateful agent workflow. It assembles instructions and context, coordinates model and tool calls through the agent loop, maintains session state, and applies configured approvals and observability. Because the harness mediates these interactions, it is a natural integration point for policy controls: Agent Hooks defines standardized interception points, while the Agent Control Specification (ACS) evaluates policy at those points and returns decisions for the host to enforce. The harness is not an execution-isolation boundary; tools and code may run in separate containers, sandboxes, Cloud PCs, or hosted environments that require their own security controls.

Models

Foundation and custom models should be governed through approved model catalogs, deployment policies, evaluation pipelines, content safety controls, and monitoring. Microsoft Foundry provides a governed environment for model selection, evaluation, deployment, and operations.

Model choice should be treated as a policy decision, not an uncontrolled developer preference. Different workloads may require different tradeoffs across cost, latency, quality, data handling, region, safety, and compliance. Enterprises need visibility into which models are used, for which workloads, and under which controls.

Azure AI Content Safety, available through Microsoft Foundry capabilities, helps protect AI applications from harmful content, prompt attacks, jailbreak attempts, unsafe outputs, and other content risks. Content safety should be treated as part of the AI control plane, not a bolt-on filter at the edge.

Agents

Agents need their own lifecycle: discovery, ownership, identity, permissions, classification, monitoring, and retirement. Agent 365 helps organizations discover, inventory, govern, and enforce policies across their agent estate, preventing it from becoming an unmanaged shadow application layer.

This matters because agent sprawl can become the next application sprawl. Without inventory and governance, organizations may not know which agents exist, who owns them, what data they access, what tools they can invoke, or whether they are still needed.

Microsoft Purview complements this by helping protect and govern the data agents use, including sensitivity labels, data loss prevention, audit, compliance, and data security posture management for AI scenarios. Microsoft Entra provides the identity fabric for users, apps, agents, and workloads, helping ensure that agent actions are tied to explicit identities and least-privilege access.

MCP, APIs, tools, and skills

The Model Context Protocol and tool-based agent architectures are powerful because they let agents connect to systems of record, business applications, knowledge stores, and automation surfaces. They are also high-value control points.

Every tool is effectively a capability grant. A read-only knowledge tool, a refund API, a database query tool, a ticketing connector, a browser automation tool, and a code execution tool carry very different risk. Tool authorization should be granular, contextual, and auditable.

Key controls include:

• Tool allowlisting and approval workflows.
• Least-privilege API permissions.
• Strong authentication through Microsoft Entra.
• Managed identities and workload identities for secretless access.
• API inspection and protection with Azure API Management and Defender for APIs.
• Network isolation with Private Link, Azure Firewall, WAF, and DDoS Protection.
• Runtime isolation with containers, Microsoft eXecution Container (MXC), Windows 365 for Agents, and confidential computing where needed.
• Monitoring and detection through Microsoft Defender, Microsoft Sentinel, and Azure Monitor.

Agent Hooks and policy gates

Because the harness coordinates the agent loop, it provides a natural host-level seam for policy enforcement. Agent Hooks defines a framework-neutral interception contract across agent input, model calls, tool calls, and output. ACS can evaluate policy at those interception points, while the conforming host applies the resulting allow, deny, transform, or approval decision.

Traditional framework callbacks are often designed for observability, not governance. They may observe events but not reliably block actions, transform content, bind approvals, or produce enforceable evidence. Agent Hooks introduces a framework-neutral governance contract for AI agents with defined interception points across the lifecycle, including startup, input, model calls, tool calls, output, and shutdown.

Agent Hooks and the Agent Control Specification are open, design-partner-informed control contracts for agent hosts and frameworks. They define where policy decisions can be requested and what obligations a conformant host has when a verdict requires blocking, approval, transformation, or record generation.

If a policy denies a tool call, the tool should not execute. If a post-tool response is denied, it should not enter agent state. If approval is required, the approval should bind to the exact action and content that was reviewed, not loosely to a session.

Agent Hooks complements, but does not replace, runtime isolation. It helps govern cooperative agent host; it does not sandbox the host, tools, or untrusted code. It therefore works alongside the Agent Control Specification, Microsoft Foundry control plane, Agent 365, Microsoft Agent Framework, Microsoft eXecution Container (MXC), and Windows 365 for Agents to combine policy governance with secure execution.

RAG and knowledge grounding

Agents become more useful when they can ground reasoning in enterprise and external context. But grounding also expands the agent’s trust boundary by introducing additional data sources, retrieval paths, indexes, and content into the reasoning process. Securing and governing these grounding sources is addressed in the next section as part of the data and context layer.

3. Govern data and context

The data and context layer is where enterprise knowledge becomes available to AI applications and agents and where existing data-security and governance boundaries must continue to hold.

Agents may ground their reasoning across enterprise data, Work IQ, Fabric IQ, Foundry IQ, Web IQ, Azure AI Search, Microsoft Graph, and other organizational knowledge sources. These sources can provide valuable context, but each introduces its own authorization model, data boundary, retrieval path, and governance requirements. Trust therefore depends not simply on connecting agents to more information, but on ensuring that grounding does not create a new path around the controls protecting that information.

Securing grounding starts with identity and authorization. Retrieval should preserve the permissions of the underlying source wherever supported and ensure that users, agents, applications, and workload identities can access only the information required for the task. Microsoft 365 Copilot respects Microsoft 365 user permissions and supported Microsoft Purview protections; other grounding architectures require source-specific configuration and validation. Access through Microsoft Graph, Azure AI Search, Microsoft Fabric, Microsoft Foundry, or other retrieval mechanisms should similarly be designed around explicit identities, least privilege, and the authorization boundaries of the underlying data.

Data protection must extend through the grounding pipeline. Sensitive information can exist not only in source documents and databases, but also in indexes, retrieved passages, embeddings, prompts, agent memory, model responses, logs, and downstream actions. Applicable sensitivity labels, information protection, DLP, retention, audit, encryption, and compliance controls should therefore be considered across the complete flow of grounded information rather than only at the original data source.

Grounding also introduces risks that traditional access control alone does not address. Overshared content can become overshared AI context; stale or low-quality information can influence agent decisions; compromised or untrusted sources can introduce poisoned content; and instructions embedded in retrieved content can attempt to manipulate agent behavior through indirect prompt injection. Organizations should therefore govern which sources can be used for grounding, validate the trustworthiness and freshness of those sources, protect retrieval and indexing pipelines, and apply appropriate evaluation, monitoring, and content-safety controls.

For retrieval-based architectures, indexes and knowledge stores should be treated as governed enterprise assets. Organizations should understand what information is indexed, where it originated, how frequently it is refreshed, which identities can query it, how access controls are enforced, and whether retrieved content can cross security or data boundaries. Retrieval relevance alone is not sufficient; the retrieval path must also preserve the security intent of the source system.

This is particularly important for Microsoft 365 grounding. Microsoft Purview helps protect and govern sensitive information through capabilities such as information protection, DLP, audit, retention, compliance, and data security posture management. Microsoft Entra provides identity and access controls for users, applications, agents, and workloads. SharePoint Advanced Management can help organizations address oversharing and content-governance risks in the SharePoint estate used by Microsoft 365 Copilot and agents. Other grounding environments require equivalent controls appropriate to their data sources, identities, retrieval architecture, and runtime.

The same principles apply across Work IQ, Fabric IQ, Foundry IQ, Web IQ, enterprise data, Azure AI Search, and Microsoft Graph, but the controls should not be assumed to operate identically across them. Their integration models, permissions, governance capabilities, and maturity differ. Organizations should validate the security boundary of each grounding source and explicitly determine how identity, authorization, data protection, retrieval, monitoring, and audit requirements are enforced for that implementation.

A trusted grounding architecture should therefore answer several questions:

• Who or what is requesting the information?
• Is that identity authorized to retrieve it?
• Is the source approved and trusted for grounding?
• Does retrieval preserve the source system's access boundaries?
• Is sensitive information protected throughout retrieval, indexing, reasoning, memory, and output?
• Can retrieved content introduce malicious or untrusted instructions into the agent?
• Are indexes and knowledge stores governed, current, and appropriately isolated?
• Can organizations observe and audit how grounded information is being accessed and used?

The principle is straightforward: grounding should extend enterprise context to agents without weakening the identity, security, privacy, and governance boundaries of the underlying information. A trusted agent should retrieve not simply the most relevant information, but the right information, from trusted sources, through authorized paths, under controls the organization can govern and audit.

4. Secure the build continuum: no-code, low-code, and pro-code

Microsoft’s platform supports multiple build paths: Agent Builder for no-code creation, Copilot Studio for low-code agents and orchestration, and Microsoft Foundry for pro-code AI systems. AI solutions can cross boundaries, and security must travel across that continuum.

No-code: Agent Builder

No-code agents make AI creation accessible to more users, which increases business agility and helps organizations bring AI into the flow of work. The trust requirement is to make that creation visible and governable without removing the simplicity that makes no-code valuable.

Microsoft Agent 365 helps provide agent discovery, inventory, lifecycle management, policy enforcement, and governance across the agent estate. Microsoft Purview helps ensure that the data used by these agents remains protected by sensitivity labels, DLP, audit, and compliance controls. Microsoft Entra helps enforce identity, access, and least privilege. Together, these capabilities allow organizations to empower business users while maintaining enterprise trust.

Low code: Copilot Studio

Copilot Studio brings orchestration, connectors, workflows, and extensibility. That makes Power Platform governance important because low-code agents often sit close to real business processes and enterprise data.

Power Platform DLP policies, Managed Environments, environment strategy, connector governance, solution lifecycle management, maker and admin roles, and approval patterns for sensitive actions are the mechanisms that turn low-code agility into trusted enterprise automation.

These controls help organizations decide which connectors can be used together, which environments are appropriate for development and production, who can build and publish agents, how solutions move through lifecycle stages, and when human approval is required before action is taken.

Low-code agents need both business speed and enterprise control. Trust is what allows organizations to scale low-code innovation beyond isolated pilots.

Pro-code: Microsoft Foundry

Pro-code agent systems demand rigorous software engineering: secure architecture and coding, dependency governance, CI/CD safeguards, container security, infrastructure-as-code scanning, environment isolation, observability, evaluation, and incident response.

GitHub Advanced Security and Microsoft Defender for DevOps are central to this approach. GitHub Advanced Security supports code and secret scanning, dependency review, and software supply-chain protection. Defender for DevOps brings these findings into Microsoft Defender for Cloud, giving security teams a unified view of AI application, cloud, and workload risks.

AI-specific assurance should also be embedded in the development lifecycle. Open-source tools such as PyRIT, RAMPART, ASSERT, and the Agent Governance Toolkit support red teaming, behavioral evaluation, governance evidence, and regression testing. Open specifications such as Agent Hooks and the Agent Control Specification establish interoperable control patterns across agent hosts and frameworks.

These open-source tools give organizations transparent, extensible ways to test and standardize AI security across heterogeneous environments. For enterprises using multiple frameworks, models, clouds, APIs, and orchestration approaches, PyRIT, RAMPART, ASSERT, and the Agent Governance Toolkit provide practical methods to red-team, evaluate, govern, and validate systems. Agent Hooks and the Agent Control Specification complement them with a common language for integrating controls at the host level.

Microsoft commercial solutions operationalize these controls at enterprise scale. Microsoft Agent 365, Microsoft Foundry, Microsoft Entra, Microsoft Purview, Microsoft Defender, Microsoft Sentinel, Microsoft Security Copilot, GitHub Advanced Security, and Azure infrastructure services provide the inventory, policy enforcement, data governance, monitoring, detection, response, compliance, and operational capabilities required for production environments.

The roles are complementary: open-source tools promote transparency, interoperability, and technical assurance, while commercial solutions provide enterprise control planes, integrated operations, support, and scale. Together, they help customers and partners secure AI systems from experimentation through production.

5. Secure runtime, isolation, and execution

Once agents are built, they need somewhere to run. The runtime layer determines how agents are isolated, monitored, scaled, and connected.

Key runtime and infrastructure components include Microsoft Foundry, Azure AI Search, Azure Confidential Computing, Azure Kubernetes Service, Azure Container Apps, Azure Functions, Azure Private Link, Storage, Azure Firewall, WAF, DDoS Protection, Azure Monitor, Azure Policy, Windows 365 for Agents, and Microsoft eXecution Container (MXC).

AI applications and agents need more than access to a model. They require secure infrastructure to host orchestration, retrieve knowledge, execute tools, process files, automate workflows, connect privately to enterprise systems, maintain state, enforce policy, and capture telemetry. This foundation turns prototypes into resilient, governable production systems.

Strong infrastructure controls are especially important because agents may execute code, invoke tools, operate browsers, process files, and interact with systems that were not designed for autonomous actors.

Microsoft eXecution Container (MXC) and Windows 365 for Agents are especially important for secure execution. Microsoft eXecution Container (MXC) – (in early preview now) is a sandboxed code-execution capability designed to provide isolated execution environments for agent actions and code. Windows 365 for Agents provides managed Cloud PC environments for agents that need to interact with desktop applications, browsers, or legacy systems that do not expose APIs.

Azure Confidential Computing can help protect sensitive workloads in use. Azure Private Link keeps traffic private. Azure Policy enforces guardrails. Azure Monitor and Application Insights provide telemetry. Azure Key Vault and Managed HSM protect secrets and keys. Managed identities and Microsoft Entra Workload ID reduce the need for long-lived secrets. Azure Container Registry and Defender for Containers help protect image-based agent workloads.

As agent autonomy and tool access increase, runtime isolation, oversight, and monitoring must strengthen accordingly.

6. Manage runtime security posture

Runtime security posture provides a continuous view of whether AI applications, agents, tools, and infrastructure remain within expected security boundaries. Layer 5 defines where and how agents run; Layer 6 assesses whether those environments are hardened, observable, compliant, and improving over time.

For agentic systems, posture management must combine traditional cloud controls with AI-specific oversight. Security teams need visibility into AI services, agent workloads, containers, APIs, model endpoints, grounding stores, identities, secrets, and network paths. They must also identify public exposure, missing telemetry, excessive permissions, vulnerable images or packages, unprotected data stores, and access to unapproved tools or data sources.

Microsoft Defender for Cloud anchors this layer. Cloud Security Posture Management identifies misconfigurations, excessive permissions, exposed resources, weak network paths, encryption gaps, and compliance issues across cloud workloads. Defender for Containers extends that visibility to images, registries, clusters, and runtime risks. Defender for AI services threat protection and AI Security Posture Management add AI-specific discovery and protection, surface risky configurations, and connect those findings to broader cloud and workload risk.

Effective runtime posture also requires identity and data context. Microsoft Entra, managed identities, Entra Workload ID, Conditional Access, and Privileged Identity Management help reduce standing privilege and reliance on secrets. Microsoft Purview adds sensitivity, DLP, audit, retention, compliance, and data-security posture signals. Agent 365 and Microsoft Foundry add context about agent inventory, ownership, evaluations, guardrails, model endpoints, and lifecycle state.

The goal is not to generate more recommendations, but to prioritize the risks that matter most in production: exposed agent endpoints, unmanaged MCP tools or APIs, overprivileged workload identities, vulnerable containers, missing telemetry, unprotected secrets, overshared grounding data, noncompliant model deployments, and runtime environments that do not match an agent’s autonomy.

These posture signals should flow into security operations. Defender XDR, Microsoft Sentinel, Microsoft Security Exposure Management, and Security Copilot can combine posture, exposure, detection, and audit data to help defenders investigate agent behavior, trace attack paths, and coordinate response. This connects build-time assurance, runtime isolation, continuous posture management, and security operations.

7. Operationalize AI security across the Microsoft Platform

AI security must extend beyond design and deployment into continuous operations. As agents move into production, organizations need to connect agent and application governance, identity, data security, cloud and workload protection, exposure management, detection, investigation, and response so agent activity can be understood and governed in the context of the broader enterprise environment.

Microsoft provides complementary control planes across this operating model. Microsoft Agent 365 and Microsoft Foundry Control Plane help govern the agent estate and AI platform; Copilot Studio and Power Platform governance provide environment, connector, and DLP controls for low-code agents and applications; Microsoft Entra governs identity and access; Microsoft Purview protects and governs data; and Microsoft Defender, Microsoft Security Exposure Management, Microsoft Sentinel, and Microsoft Security Copilot extend protection, exposure management, detection, investigation, and response across the environment.

Microsoft Security provides an integrated operating model across these domains:

For agentic systems, this connected view is especially important. An agent may authenticate with one identity, retrieve sensitive data from another system, invoke APIs or MCP tools, execute code in a runtime, and take action in a business application. Security operations therefore need to correlate relevant identity, data, application, API, cloud, endpoint, workload, and agent signals rather than assess each component in isolation.

Posture, exposure, telemetry, detections, and audit signals can then help defenders understand suspicious agent activity in context—for example, an over-permissioned identity, exposed API, vulnerable runtime, unusual data access, or anomalous agent behavior. The goal is not to create a separate security operations model for AI, but to extend existing enterprise security operations to AI applications and agents.

Defend with AI

Trust also means moving from defending AI to defending with AI. Microsoft is applying AI across security operations, software assurance, and proactive threat discovery to help defenders identify risk, investigate threats, and accelerate response.

Microsoft Security Copilot augments security teams with AI-assisted investigation, incident summarization, signal correlation, query generation, and response workflows. Codename MDASH, currently in preview, is Microsoft’s multi-model agentic scanning harness, an agentic code scanner in Microsoft Defender that coordinates specialized agents and models to discover, validate, and help remediate source-code vulnerabilities. Project Perception, currently in limited preview, coordinates red, blue, and green team agents to uncover weaknesses, investigate threats, and recommend or perform approved remediation.

Together, these capabilities demonstrate the next evolution of security operations: using security to protect AI while using AI to strengthen security combining human expertise with AI-assisted and agentic defense across prevention, exposure management, detection, investigation, and response.

Apply security foundations across every layer

Security foundations provide consistent principles that underpin trust across the AI and agent lifecycle. Zero Trust, encryption, observability, continuous monitoring, Responsible AI, and governance become actionable when they are applied as concrete controls across identity, data, applications, agents, tools, grounding, and runtime environments.

• Zero Trust: Verify explicitly, enforce least privilege, and assume breach across user, agent, workload, tool, data, and runtime access.
• Encryption: Protect prompts, responses, grounding data, memory, logs, and tool payloads at rest, in transit, and where supported, in use.
• Observability and logging: Capture relevant agent activity including tool calls, data access, policy decisions, approvals, errors, and outcomes with appropriate privacy controls.
• Responsible AI and governance: Evaluate agents for safety, reliability, privacy, security, transparency, and accountability before and after deployment.
• Continuous monitoring: Monitor for drift, misuse, anomalous behavior, unexpected data access, and emerging threats as models, data, and tools change.
• Audit and accountability: Maintain evidence of who initiated an action, which agent acted, what resources were accessed, which controls were applied, and what outcome occurred.
• Privacy and data protection: Apply appropriate information protection, DLP, retention, audit, and compliance controls across prompts, grounding, memory, outputs, logs, and downstream actions.

These controls must operate continuously across the agent lifecycle. As agents move from design and build through grounding, deployment, monitoring, and response, their models, data, tools, identities, permissions, and runtime environments will continue to change. Trust therefore cannot be established once at deployment; it must be continuously evaluated, governed, monitored, and improved as the agent and its operating environment evolve.

The outcome: trusted autonomy

Trust is not a constraint on AI transformation; it is what enables AI to scale responsibly across the enterprise.

As organizations move from assistive copilots to increasingly autonomous agents and multi-agent systems, they need confidence that those systems operate within clear boundaries for identity, data access, tool use, execution, governance, and accountability. The goal is not to eliminate autonomy, but to make autonomy intentional, bounded, observable, and governable.

Microsoft brings together AI platforms with identity, data security, application security, cloud and runtime protection, security operations, and open security tooling to help organizations establish these boundaries across the agent lifecycle. Rather than creating a separate security model for every agent, framework, runtime, or data source, organizations can apply a consistent trust architecture while adapting controls to the risk and autonomy of each scenario.

This is trusted autonomy: enabling agents to reason and act with increasing independence while keeping their access, actions, and outcomes within enterprise-defined security and governance boundaries.

Combining the intelligence that makes agents useful with trust that allows organizations to deploy them confidently at enterprise scale.

Developers are shipping agents faster than security teams can catalog them. As organizations move beyond pilots and begin operating dozens or hundreds of agents, one question keeps coming up: how do we actually get visibility into our AI agents across our environment? 

In this article, we'll walk through how Azure services can help establish visibility, guardrails, and cost accountability across your AI estate.

Governance for AI happens across four layers:

1. Resources – who can create new AI resources
2. Builders – who can develop and publish agents in a certain scope
3. Behavior – how agent outputs are evaluated, monitored, and governed
4. Dependencies – what models, tools, APIs, and MCP servers agents can interact with

Most organizations already have governance controls for identities, networking, and compliance. The challenge isn't creating new controls. It's connecting existing controls into an operating model that works for AI agents.

Below we walk through each area and go a bit deeper on how to close the gap.

Setting up boundaries with Azure Policy

First, let's start in the Azure portal with Azure Policy. Azure Policy lets you set guardrails on what can be deployed in your environment and flags or blocks anything that doesn't comply. For AI workloads, the built-in definitions range from limiting models that people in your organization can deploy to locking down the network through enabling private endpoints.

Some policies you get started with:

• Foundry model deployments should only use approved models: Restricts deployments to models or publishers your organization has explicitly approved
• Foundry model deployments should meet eligibility requirements (preview): Applies rules based on model attributes like preview vs. GA status and distribution source
• Azure AI Services resources should have key access disabled: Makes Microsoft Entra ID the only entry point 

The full list of policies related to Azure AI Services are available here: List of built-in policy definitions - Azure Policy | Microsoft Learn

Why this comes first: Policy checks resources before they are created, so it proactively keeps your environment aligned with your standards.

Implementing role-based access control (RBAC)

Once these boundaries are in place, the next step is RBAC. Setting RBAC up early ensures that people and identities building agents have the right scope for what they actually need to do. Foundry roles only apply when you authenticate using Microsoft Entra ID. If you're using key-based authentication instead, the key grants full access with without role restrictions. API keys are convenient for quick development usage but when moving towards production, Microsoft Entra ID is the preferred method.

Roles can be assigned at three scopes: the Foundry resource, a Foundry project, or an individual agent itself. Below is an example of how different personas within organization can map to a certain scope for creating and building agents with Foundry.

Here's how each role in the diagram compares, from least to most privileged:

Source: Role-based access control for Microsoft Foundry - Microsoft Foundry | Microsoft Learn

For the agent resources themselves, assign managed identities rather than API keys since it lowers the risk of having compromised credentials.

Note if you're scripting RBAC permissions: these roles were recently renamed from Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. The role IDs and permissions didn't change, so use the role definition GUID in your code to avoid issues while the rename rolls out.

Observability in Microsoft Foundry

Governance requires more than access control. Organizations also need evidence of how agents are being used. Observability provides the audit trail needed to investigate incidents, understand usage patterns, and track costs. The Foundry Control Plane brings these observability and governance tools together in one place, alongside services like Azure Monitor, Microsoft Entra, Microsoft Purview, and Azure Policy.

In the Foundry portal, tracing is a good starting point. Once you connect an Application Insights resource to your project, Foundry turns on tracing automatically so every run, including the ones you test in the playground, is logged. After it is completed, you can search by Response ID or Trace ID to see the conversation history, token usage, run steps, tool calls, and inputs and outputs between the user and the agent. For more granular queries, you can write KQL to dive into individual agent runs or use the prebuilt Grafana dashboards in Azure Monitor. With client-side tracing, you can also export traces to observability tools you may already use, such as Datadog or Jaeger.

Note: Permissions required for viewing this telemetry requires the Log Analytics Reader role on the connected Application Insights resource, and Privileged Monitoring Data Reader on top of that if the underlying Log Analytics tables are protected.

Alongside all of these monitoring features, every agent comes with content safety guardrails and evaluations let you test and optimize performance before and after you publish your agents.

When agents get published to Microsoft Teams and Microsoft 365 Copilot, Microsoft 365 admins can approve usage requests. These requests can be further scoped to a limited group for pilot testing/department usage or the full organization.

Adding an AI gateway

Observability tells you what your agents are doing, but how do you actually control them? This is where Azure API Management comes in. Once you have more than one agent, model deployment, or multiple teams consuming them, you need a single enforcement point between the agents and the resources they call. Adding Azure API Management in front of Microsoft Foundry gives you:

• Rate limiting and load balancing across regions and model deployments

• Consistent authentication and quota policies for model and tool traffic

• Usage tracking per team or cost center so you can accurately charge back to different departments
• Governed access to your custom and remote MCP servers

Note: When choosing MCP servers, start with trusted, enterprise supported sources (GitHub, Microsoft, internally developed servers, etc.) that have documented security controls, enterprise authentication, clear ownership, and least-privilege permissions. Treat community MCP servers as untrusted until they have undergone a formal security review and are verified by organizations.

Adding an AI gateway completes the governance picture. Azure Policy governs what can be deployed. RBAC governs who can build and manage agents. Observability provides evidence of how agents behave in production. The AI gateway extends governance into runtime, controlling how agents interact with models, tools, and external systems. Combined, these layers help organizations move beyond simply building agents to operating them responsibly at scale.

Extend governance across the wider estate

A few directions to take this further:

• MCP registry in Azure API Center – As MCP usage grows, you can create an approved inventory of MCP servers and APIs that can be used across an organization.
• Microsoft Agent 365 – Microsoft's enterprise control plane for AI agents. It gives every agent its own Microsoft Entra Agent ID and published Foundry agents sync to its registry automatically. This gives your IT team one place to run access reviews, lifecycle policies, and owner attestation across every agent in the tenant, including shadow agents discovered outside Foundry.
• Copilot Studio – when you add an MCP tool, you can point it at the API Management URL instead of the direct remote endpoint to gain additional observability through the gateway.
• GitHub Copilot – you can apply the same AI gateway-fronted MCP registry, applied to the developer side.
• Microsoft Purview – data classification, DLP, audit, and AI interaction governance across the wider estate.

Where to go next

• Looking for a quick start? Turn on the three Azure Policy definitions above in audit mode against a non-production subscription and see what gets flagged that is out of compliance.
• Ready to design the end-to-end pattern? Take a look at this Cloud Adoption Framework guidance on AI governance and governing Azure platform services for AI.
• Want to go deeper on agent observability? Start with these articles around Application Insights integrations with Foundry: Use Insights in Microsoft Foundry and Monitor AI Agents with Application Insights

Governing AI agents doesn't require starting from scratch. The identity, policy, monitoring, and cost controls you already use for the rest of your Azure estate can extend to AI workloads. Start with one layer, connect the next, and build a governance foundation that grows with your AI adoption.

The Microsoft AI Tour kicks off on October 12 and will make its way around the world with stops in key markets where business decisions are made. Become a sponsor to share your AI innovation with qualified audiences during these one-day events designed to help organizations move from AI experimentation to confident action. 

Visit Microsoft Event Sponsorship or contact sponsor@microsoft.com to get the 2026–2027 Microsoft AI Tour sponsorship opportunity guide. 

As a Microsoft sponsor, you can: 

• Generate pipeline—make meaningful, face-to-face connections with decision-makers and practitioners across industry segments. 

• Build brand visibility—boost your brand’s profile and strengthen your alignment with Microsoft and AI. 

• Showcase your innovations—share our latest joint solutions in your booth and during sessions to spark interest and gain leads. 

Reach out to sponsor@microsoft.com to start planning your sponsorship today. 

Where your Oracle data meets Microsoft AI  

Most enterprise AI runs into the same wall: the data lives in Oracle, and the AI lives somewhere else. Oracle AI Database@Azure closes that gap, running Oracle data alongside Microsoft’s trusted AI platform.  

Microsoft is a sponsor at Oracle AI World 2026 — register now to reserve your seat and stop by booth #10028 to talk to experts and win prizes! 

Hear from Microsoft’s Brett Tanzer 

Four Clouds, One Conversation: OCI, AWS, Azure, and Google Leaders on What's Next  [LRN1134]  Join Microsoft Vice President Brett Tanzer for a four-cloud panel discussion exploring what’s next for multicloud data and AI, and why the decisions being made today will shape tomorrow’s cloud strategy.   October 26 | 11:30AM

Microsoft sessions you won't want to miss 

 Check out additional sessions featuring Microsoft and Oracle product experts.  

Stop by the Microsoft booth (#10028) in the hub – where innovation meets fun! 

Connect with Microsoft and explore how Oracle AI Database@Azure can help accelerate your cloud strategy and unlock new possibilities with Microsoft AI. We're running live demos with our experts throughout the day, so come see the platform in action and bring the questions you can't get answered anywhere else. Enter our raffle for a chance to win– all while exploring the most innovative Azure solutions driving enterprise transformation.