At first glance, displaying data in a web application seems simple. Render a table, load some rows, and you’re done.

But that simplicity rarely survives contact with real-world requirements.

Users eventually ask for filtering, inline editing, exports, grouped views, responsive layouts, role-based access, keyboard navigation, localization, and the ability to handle millions of records through virtualization, paging, incremental loading, and server-side data operations.

That’s why choosing a JavaScript data grid isn’t just a UI decision. The right grid can accelerate development and improve user productivity. The wrong one can create performance issues, increase maintenance costs, and slow down future product development.

In this guide, we’ll compare five popular JavaScript data grid libraries in 2026:

• AG Grid
• TanStack Table
• Syncfusion JavaScript DataGrid
• Handsontable
• Kendo UI Grid

Rather than looking for a universal winner, we’ll focus on which grid works best for specific application requirements.

Feature	AG Grid	TanStack	Syncfusion	Handsontable	Kendo
Virtualization	Yes	API-driven	Yes	Yes	Yes
Excel Export	Yes	Custom	Yes	Yes	Yes
PDF Export	Yes	Custom	Yes	Limited	Yes
Built-in Editing	Yes	Custom	Yes	Yes	Yes
Headless	No	Yes	No	No	No

How we evaluated these JavaScript Data Grids 

This comparison is based on publicly available product documentation and capabilities reviewed in July 2026. We evaluated each library based on its data-handling model, built-in features, customization options, developer experience, accessibility support, licensing approach, ecosystem maturity, and suitability for common application scenarios. 

The order does not represent a universal ranking. Each grid has different strengths, and the best choice depends on your application’s workflows, technical requirements, framework, budget, and long-term maintenance needs. Product capabilities and licensing terms can change, so verify the latest vendor documentation before making a final decision. 

Feature availability, APIs, licensing, and accessibility details may also differ across framework-specific editions. Verify the documentation for the exact package and edition your application uses. 

Why choosing the right JavaScript Data Grid matters

In many business applications, the data grid becomes the primary workspace for users. Instead of simply viewing data, users filter records, edit values, export reports, review exceptions, and complete day-to-day operational tasks directly within the grid.

This is common in applications such as:

• Admin dashboards
• ERP and CRM systems
• Inventory management software
• Financial reporting tools
• Analytics platforms
• Data review and approval workflows
• Internal operations portals
• B2B SaaS applications

The challenge is that not all JavaScript data grid libraries are designed for the same purpose.

Some provide a fully featured component with built-in capabilities such as editing, filtering, grouping, and export. Others focus on table logic and give developers complete control over the user interface. There are also solutions designed specifically for spreadsheet-style workflows and data-entry scenarios.

Because the grid often becomes one of the most frequently used parts of an application, choosing the right solution can affect development effort, performance, maintainability, and overall user experience. Before comparing individual libraries, it’s worth understanding the different approaches available and how they align with your application’s requirements.

Three approaches to JavaScript Data Grids

Before comparing products, it’s useful to understand the three major categories of grid solutions.

Component-based grids

Component-based grids ship with built-in UI, interaction models, styling systems, and common data operations.

They’re typically chosen by teams that want enterprise-grade functionality without having to build every interaction from scratch.

Common examples:

• AG Grid
• Syncfusion DataGrid
• Kendo UI Grid

These solutions work particularly well for business applications, reporting portals, ERP systems, CRM platforms, and operational dashboards.

Headless grid libraries

• Headless libraries focus on data-processing logic rather than visual presentation.
• Instead of providing a complete UI, they offer APIs and state management that let you build your own experience.
• TanStack Table is a widely used example of this approach.
• This model works well for organizations with established design systems and teams that require complete UI control.
• In return for that control, the development team is responsible for implementing or integrating the rendered UI, styling, semantic markup, accessibility, focus management, and keyboard interaction behavior required by the application.

Spreadsheet-oriented grids

Some applications revolve around data entry rather than data presentation.

Spreadsheet-style grids provide cell editing, keyboard workflows, copy-paste support, validation, and interactions that mimic spreadsheet software.

Handsontable is a popular candidate in this category.

How to evaluate a JavaScript data grid library

When comparing grid libraries, feature count alone is rarely the deciding factor for long-term success. Most mature products already support filtering, sorting, paging, and editing. 

The more important question is whether the available features match your workflows, performance requirements, customization needs, and maintenance capacity. 

Instead, focus on these questions:

Performance

How will the grid behave with realistic datasets?

Consider:

• Large record counts
• Frequent updates
• Virtual scrolling
• Server-side data operations

Editing requirements

Do users need:

• Inline editing?
• Batch updates?
• Validation rules?
• Custom editors?

Customization

How closely must the grid match your design system?

Some products provide extensive customization, while others prioritize ready-made functionality.

Accessibility

For many enterprise applications, accessibility is a requirement rather than an option. Verify the grid against your organization’s required WCAG 2.1 or WCAG 2.2 conformance level, and test the configured implementation rather than relying only on a vendor-level accessibility statement. 

Evaluate:

• Keyboard-only navigation and operation 
• Logical and visible focus management 
• Screen-reader behavior 
• Accessible editing, validation, and error reporting
• Roles, names, states, and relationships 
• Alternatives for drag-based interactions 
• Behavior after customization and theming 

Export and reporting

Many business workflows still depend on:

• Excel export
• CSV export
• PDF export

If reporting is critical, ensure these capabilities meet your requirements.

Licensing and support

Before committing, verify:

• Commercial licensing terms
• Production restrictions
• Support availability
• Upgrade policies

JavaScript Data Grid libraries at a glance

Library	Best For	Primary Strength
AG Grid	Data-intensive enterprise applications	Extensive feature set and scalability
TanStack Table	Design-system-driven products	Complete UI flexibility
Syncfusion DataGrid	Business applications and dashboards	Broad enterprise functionality out of the box
Handsontable	Spreadsheet-style workflows	Familiar editing experience
Kendo UI Grid	Existing Telerik ecosystems	Integration with a larger UI platform

Detailed Review of the Top JavaScript Data Grid Libraries

1. AG Grid

AG Grid is widely adopted in enterprise web applications and is frequently evaluated for data-intensive use cases. Its extensive feature set, multiple row models, and support for popular JavaScript frameworks make it a strong candidate for applications with complex data requirements. 

One of AG Grid’s biggest strengths is its ability to handle complex requirements without relying heavily on additional libraries. Features such as filtering, grouping, aggregation, pivoting, virtualization, and advanced data operations make it a strong candidate for large-scale business applications.

Key strengths

• Extensive feature set for enterprise scenarios
• Multiple row models for client-side and server-side data scenarios
• Mature ecosystem and documentation
• Community and Enterprise editions to suit different project needs
• Works across multiple JavaScript frameworks

Things to consider

AG Grid’s breadth of functionality can be more than smaller projects require. Teams should also carefully review feature availability between the Community and Enterprise editions before making a long-term commitment.

Ideal use cases

AG Grid is particularly well suited for reporting systems, operational dashboards, financial applications, analytics platforms, and enterprise software where users spend significant time working with data.

2. TanStack Table

Unlike traditional grid components, TanStack Table takes a headless approach. Instead of providing a ready-made user interface, it focuses on the underlying table logic and state management while giving developers complete control over rendering and styling.

This flexibility makes it suitable for teams that maintain their own design systems or need highly customized user experiences. Rather than adapting a prebuilt grid to match an application’s design, developers can build exactly the experience they need.

Key strengths

• Complete control over UI and styling
• Headless architecture with minimal visual constraints
• Well suited for integration with custom design systems
• Supports multiple frontend frameworks
• Composable row models and controlled state management

Things to consider

This flexibility comes with additional implementation responsibility. TanStack Table provides the underlying table logic rather than a complete grid interface, so developers must implement or integrate the UI rendering, styling, semantic markup, accessibility, focus management, and keyboard interactions required by their application. Features available out of the box in component-based grids may therefore require additional development, integration, and testing.

Ideal use cases

TanStack Table is a strong choice for SaaS applications, internal platforms, design-system-driven products, and teams that prioritize flexibility over out-of-the-box functionality.

3. Syncfusion JavaScript DataGrid

Syncfusion JavaScript DataGrid is a feature-rich HTML5 data grid component for displaying and managing tabular data in JavaScript applications. It includes data binding, editing, Excel-like filtering, custom sorting, aggregation, selection, and export support for Excel, CSV, and PDF formats.

It is designed for applications where the grid supports real business workflows, not only data display. Teams can bind data from local and remote data sources, including JavaScript object arrays, JSON, OData services, REST APIs, and other supported endpoints through DataManager.

Key strengths

• Broad built-in functionality for data presentation and editing 
• Local and remote data binding through DataManager 
• Filtering, grouping, sorting, summaries, and selection 
• Export options for Excel, CSV, and PDF.
• Responsive layouts, themes, and localization support 

Things to consider

Applications that require complete control over markup and interaction behavior may prefer a headless solution. Teams should also evaluate licensing, package size, required modules, framework compatibility, and whether the broader component suite aligns with their project needs.

Ideal use cases

Syncfusion JavaScript DataGrid works well for ERP modules, CRM systems, inventory management applications, business dashboards, reporting portals, and enterprise applications with complex data workflows.

4. Handsontable

Handsontable approaches data grids from a different angle. Rather than focusing primarily on data presentation, it emphasizes spreadsheet-style interaction and editing experiences.

Users can work directly with cells, making it a natural fit for workflows where entering, validating, and manipulating data is more important than simply viewing it.

Organizations migrating spreadsheet-based processes to web applications often find Handsontable particularly appealing because it provides familiar interaction patterns that reduce the learning curve for end users.

Key strengths

• Spreadsheet-like user experience
• Cell-based editing workflows
• Familiar copy-and-paste interactions
• Validation and structured data-entry capabilities
• Framework support across modern JavaScript ecosystems

Things to consider

Not every application benefits from spreadsheet-style behavior. If users primarily view data rather than actively edit and manipulate it, a traditional data grid may be a better fit.

Ideal use cases

Handsontable is well suited for planning tools, budgeting systems, resource management applications, inventory workflows, financial data-entry screens, and other spreadsheet-oriented business processes.

5. Kendo UI Grid

Kendo UI Grid is part of the broader Kendo UI ecosystem and is often chosen by organizations that want a consistent set of UI components across their applications.

The grid provides a mature collection of data management capabilities, including editing, filtering, sorting, grouping, export, and both local and remote data operations. Its integration with the wider Kendo UI platform can simplify development for teams already invested in the Telerik ecosystem.

Key strengths

• Mature and comprehensive grid functionality
• Multiple editing modes
• Local and remote data binding
• Export support for business reporting workflows
• Strong integration with other Kendo UI components

Things to consider

The platform delivers the greatest value when organizations use other Kendo UI components alongside the grid. For projects seeking a lightweight standalone solution, the broader ecosystem may introduce functionality that isn’t required.

Ideal use cases

Kendo UI Grid is a practical choice for business applications, enterprise portals, administrative systems, and organizations that have standardized on Telerik and Kendo UI technologies.

Notable alternatives worth evaluating

The JavaScript grid ecosystem continues to evolve. Depending on your requirements, other libraries may also deserve consideration:

• Tabulator
• Grid.js
• MUI Data Grid
• React Data Grid

These solutions can be attractive for teams looking for lighter-weight implementations or framework-specific experiences.

Production-readiness checklist

Before making a final decision, test your shortlisted grids using realistic workloads.

Validate:

• Initial grid render time 

• Time until the grid becomes interactive 

• Scroll responsiveness and dropped frames 

• Sorting, filtering, grouping, and editing response times 

• Row and column virtualization behavior 

• Memory consumption during prolonged use 

• Performance during frequent data updates 

• Bundle-size impact 

• Server request volume and payload size 

• Paging, caching, and request-cancellation behavior 

• Export time and memory use for realistic datasets 

• Keyboard and screen-reader workflows 

• Mobile and responsive behavior 

• Licensing limitations 

• Documentation quality 

• Upgrade and long-term support expectations 

Run these tests with production builds, realistic row and column counts, representative cell templates, expected network conditions, and the same browser and device profiles used by your target audience. 

Feature comparison tables are useful, but production testing often reveals the best choice faster than any marketing page.

Conclusion

The best JavaScript data grid in 2026 depends entirely on how your users work with data.

• Choose AG Grid for feature-rich, data-intensive applications.
• Choose TanStack Table when complete UI control matters most.
• Choose Syncfusion JavaScript DataGrid when you need enterprise functionality out of the box.
• Choose Handsontable for spreadsheet-style workflows.
• Choose Kendo UI Grid when you’re already invested in the Telerik ecosystem.

Instead of selecting the grid with the longest feature list, focus on the one that aligns with your application’s workflows, performance requirements, and long-term maintenance goals. That decision will deliver far more value than any single feature ever could.

Try the Syncfusion JavaScript DataGrid 

If you’re a Syncfusion user, you can download the setup from the license and downloads page. Otherwise, you can download a free 30-day trial.

You can also contact us via our support forum, support portal, or feedback portal for queries. We are always happy to assist you!

 

Last time, we wrote a wrapper delegate that checked whether the context it was being invoked from matched the context it was captured from.

    if (d.try_as<::INoMarshal>()) {
        return [d = std::forward<Delegate>(d),
                context = winrt::capture<IContextCallback>(CoGetObjectContext)](auto&&...args) {
            if (context == winrt::capture<IContextCallback>(CoGetObjectContext)) {
                d(std::forward<decltype(args)>(args)...);
            } else {
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
            }
        };
    }

We did this by comparing context objects.

This obtains the current object context in order to compare it with the original one, and that means an internal Add­Ref, and then we have to explicitly Release it.

But there’s a way to do this without having to obtain any objects.

The Co­Get­Context­Token function gives you an integer that uniquely identifies a live context object. You can then compare integers instead of having to compare COM objects.

Note that the context must be live. Once you allow the context to destruct, the value might be reused. (You’re already used to this. Process and thread IDs work the same way: They remain unique as long as they are running or you still have a reference to them by a HANDLE.)

Since we are keeping the context alive by the IContext­Callback returned by Co­Get­Object­Context, we can pair that with a context token to make for faster checks in the future.

ULONG_PTR get_context_token()                       
{                                                   
    ULONG_PTR token;                                
    winrt::check_hresult(CoGetContextToken(&token));
    return token;                                   
}                                                   

    if (d.try_as<::INoMarshal>()) {
        return [d = std::forward<Delegate>(d),
                context = winrt::capture<IContextCallback>(CoGetObjectContext),
                token = get_context_token()](auto&&...args) {
            if (token == get_context_token()) {
                d(std::forward<decltype(args)>(args)...);
            } else {
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
            }
        };
    }

Are we done?

Of course not!

There’s a flaw in the above code. More next time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 4 appeared first on The Old New Thing.

First-of-its-kind telecom AI deployment

Telecommunications organizations are increasingly looking to AI to help teams navigate highly specialized domains, but generic models often lack the industry-specific knowledge needed to understand telecom networks, standards, and operations. To address that gap, AT&T created their Open Telco (OTel) models, the next generation of telecom-focused AI designed to bring deeper telecommunications expertise into AI systems. Building OTel2.0 required more than training a large language model, it reflected a broader issue many organizations face: how to build domain-specific AI systems at scale while balancing cost, performance, and operational complexity. Cost management quickly became a key consideration. To continue advancing telecom-focused AI, AT&T needed a platform capable of supporting OTel2.0 development at an entirely new scale.

Where teams previously had to own and manage deployments, infrastructure, and the associated operational overhead, Foundry Managed Compute provided a more streamlined way to access dedicated graphics processing unit (GPU) capacity. This transformation requires more than powerful models; it requires the ability to scale without compromising cost, flexibility, or performance.

Using Microsoft Foundry Managed Compute, AT&T was able to experiment across multiple open models, optimize workloads across different GPU architectures, and process massive volumes of telecom data all within a unified platform. The result was an AI development environment capable of supporting trillions of tokens while giving teams the flexibility to iterate, optimize, and innovate faster.

Model choice meets infrastructure flexibility

Building OTel2.0 required flexibility across both models and infrastructure. Rather than standardizing on a single model, AT&T adopted a multi open-model strategy. Open models were central to AT&T’s approach because they provided the flexibility to work with approved telecom data, tailor the workflow for domain-specific model development, and support large-scale experimentation with greater control over cost and deployment strategy. Through Microsoft Foundry, the team deployed several models from the Hugging Face collection, including Phi-4, OSS-120B, and Gemma-4, to support different stages of development, from synthetic data generation and data preparation to reasoning-intensive workloads and broader model development efforts. Phi-4 played a significant role in this process, processing more than 700 billion tokens a month as part of the broader data preparation and training workflow for OTel2.0.

Every company in the world needs to build its own AI, and that is only possible with open models and open source. AT&T is championing this vision, building on open models like Phi-4 and Gemma, and giving OTel back to the community as a telecom AI foundation others can build upon. Microsoft Foundry makes this practical at scale, bringing the latest open models from the Hugging Face collection together with AMD and NVIDIA GPUs in one place, so teams can pick the right model and the right hardware, then deploy in hours instead of weeks.

—Jeff Boudier, Vice President of Product, Hugging Face

Developing OTel2.0 also required infrastructure capable of operating at telecom scale. AT&T used approximately 530 GPUs through Microsoft Foundry Managed Compute spanning multiple GPU architectures including 430 AMD Instinct™ MI300X GPUs. This heterogenous approach gave AT&T more flexibility in how models were deployed and optimized as requirements evolved.

Model	Example workload
Phi-4	Around 700B tokens a month for data preparation and synthetic data generation
OSS 120B	Higher-reasoning workloads
Gemma 4	OTel2.0 development workflows

This flexibility illustrates a broader trend across AI development. Organizations increasingly need platforms that allow them to choose the right model for the job, optimize for cost and performance, and scale workloads without rebuilding operational environments. Microsoft Foundry brings model choice, infrastructure flexibility, governance, and operational scale together in a unified platform that supports those requirements.

Beyond flexibility and cost, deployment speed is a critical factor for many AI initiatives. As workloads expand and new models are evaluated, the ability to access GPU capacity quickly enables teams to move from experimentation to execution faster without lengthy provisioning cycles. With Foundry Managed Compute, AT&T could deploy and scale models in days rather than waiting weeks for infrastructure to become available, helping accelerate development timelines and maintain momentum across OTel2.0 development.

Optimizing cost without limiting innovation

As AI workloads grow, economics become as important as model performance. For AT&T, one of the primary objectives was to lower AI model consumption costs while continuing to drive meaningful business value through AI-powered innovation. By using open models on Microsoft Foundry Managed Compute, AT&T was able to support large-scale data preparation and model development using a different economic model built around dedicated GPU infrastructure and open-model flexibility.

The impact became clear at scale. In support of OTel2.0, AT&T processed approximately 1T tokens, consisting of raw documents from GSMA supplemented by synthetic data generated. Generating the data using open-source models like Phi-4, served by Microsoft’s Foundry Managed Compute, saved tens of millions of dollars versus using frontier models. This allowed teams to invest in larger-scale experimentation and development while maintaining a focus on business value and operational efficiency.

Metric	Value
OTel 1.0 Downloads	Over 25M
GPUs Used Through Foundry Managed Compute	About 530
Tokens Processed for OTel2.0	About 1T
Tokens Trained for OTel2.0	About 400 B
Models used to train OTel	Phi-4, OSS 120B, Gemma 4

When you are processing hundreds of billions of tokens, infrastructure becomes part of the problem you solve. Foundry Managed Compute gave us access to GPU capacity at scale so our teams could focus on advancing OTel2.0 instead of managing infrastructure.

—Mark Austin, Vice President, Data Science and AI at AT&T

At this scale, infrastructure is no longer simply a deployment consideration. It becomes a strategic component of AI development.

Accelerating the next wave of production-scale AI

OTel 2.0 demonstrates how organizations can combine open models, scalable infrastructure, and domain expertise to build production-ready AI systems. By matching different models to different workloads and optimizing infrastructure for cost and performance, AT&T was able to process trillions of tokens while maintaining operational efficiency. 

As organizations move from AI experimentation to production deployment, they increasingly need the flexibility to choose the right models, optimize infrastructure, and scale efficiently. Microsoft Foundry and Foundry Managed Compute help support that transition by bringing those capabilities together in a unified platform.

Learn more

• Read Scott Guthrie’s blog about Azure AI and HPC infrastructure.
• Learn more about OTel2.0.

Explore session topics from AMD’s Advancing AI:

• From GPUs to CPUs: Optimizing Every AI Workload with Azure and AMD
• What’s Next for AI Infrastructure in the Cloud?

The post AT&T and Microsoft scale trillion-token workloads with Microsoft Foundry and AMD appeared first on Microsoft Azure Blog.

An employee opens the HR agent and asks, "How much PTO do I accrue each month?" A few minutes later, someone else asks, "Where is the official code of ethics policy?" Those sound like the same problem. They are not. The first person needs a grounded answer they can understand. The second person needs a link to the right document quickly, without interpretation. If you design for one experience, the other one feels broken.

That is usually where knowledge-agent projects start to get messy. “Grounding” can sound like one switch you turn on, but in practice it is a spectrum: from zero-code classic search, to agentic retrieval over a knowledge base, to a forced-grounding agent that synthesizes answers when synthesis is required. The easier way to think about it is this: who is doing the retrieval work, and what does the user need back?

This post walks through five working retrieval patterns for an “Ask HR” agent built on Copilot Studio, Azure AI Search, and Foundry IQ. Each one is running code in the companion sample repo: foundry-copilot-hr-policy-knowledge. Each has a clear “use this when,” and the five patterns share the same reusable knowledge base so you can layer them on without re-indexing. By the end, you should have a decision tree you can reuse for your own knowledge source, whether that is HR policy, product docs, or support runbooks.

Scope: companion sample for learning and experimentation, not production-ready deployment. Review the Azure Well-Architected Framework for reliability, security, cost, and operational hardening before you ship.

The scenario: one index, many front doors

Here is the setup. The sample answers employee questions from a small corpus of internal HR policy documents: PTO accrual, hiring rules, code of ethics, blood-borne pathogen procedures, and dozens more. Underneath every pattern is one foundation: an Azure AI Search index named hr-policy-index, populated by an indexer and skillset that chunk and vectorize the documents.

Patterns A, C, and the Hosted Agent query that index directly. Patterns A2 and B add a Foundry IQ knowledge base named hr-knowledge-base on top of the same index for agentic retrieval. That layering is the part to pay attention to. The retrieval assets stay separate from the orchestration layer, so you can start with the simplest pattern, prove value quickly, and move to a more capable one later without re-indexing.

Two questions that decide everything

Before we get into the patterns, it helps to define the two retrieval terms I use throughout the rest of the post:

• Classic search, index-first retrieval: one hybrid (keyword + vector) query against an Azure AI Search index, ranked and returned. Fast and predictable.
• Agentic retrieval, the knowledge base plans multiple sub-queries from the user's question, runs them in parallel, re-ranks, and merges the results before the agent composes an answer. Higher quality on complex, multi-part questions.

If you want the fuller picture of how these two approaches map to retrieval-augmented generation, the Azure AI Search team's RAG and generative AI overview walks through the trade-offs and uses a similar HR/PTO example.

Once those terms are clear, the decision tree comes down to three practical questions:

Q1: Do users need an answer or are they really trying to find the right document?
If they just need the document, stay on the locator path. If they need the policy explained or summarized, move into the answer-synthesis path.

QL: Is the content in a citation-friendly knowledge base?
For example, SharePoint content or Azure AI Search content with a reliable blob_url. If yes, Copilot Studio can usually handle this with native citation cards in Pattern A. If not, use Pattern C with the dual tool /api/lookup path so the agent can return the exact document link.

Q2: Do you actually need an LLM agent in the middle?
If the answer is no, keep it simple: use classic search or agentic retrieval over the knowledge base. If the answer is yes, move into the agent path.

QK: For that non-agent path, is classic index search enough, or do you need agentic KB retrieval?
Classic search points to Pattern A. Agentic retrieval over the knowledge base points to Pattern A2.

Q3: If you need an agent, do you want Foundry to run the request loop, or do you need to self-host it?
If Foundry can manage the runtime, use Pattern B. If you need the request loop in your own container, use the Hosted Agent.

That is the decision tree in plain terms: Q1 decides whether this is a document-locator experience or an answer-synthesis experience. Q2 decides whether you need an LLM agent at all. Q3 is only about where the agent runs, either Foundry or your container. It does not change the front door; Copilot Studio can still be the user-facing experience.

How the sample repo is organized

The repo follows the same flow as the post. Start with docs/DataPipelineAndTesting.md to understand how the HR policy corpus is indexed, tested, and validated. Use docs/RetrievalPatterns.md as the decision model for choosing between classic search, agentic retrieval, forced grounding, and hosted runtime options. Then use the pattern-specific docs when you are ready to wire each path.

For Copilot Studio patterns, docs/CopilotStudioIntegration.md maps to Pattern A, while docs/CopilotStudioHybridExample.md maps to Pattern C and the dual-tool locator flow. For the more advanced agent paths, docs/FoundryAgentArchitecture.md covers Pattern B and the hosted agent architecture. docs/Distribution-M365-Teams.md shows how the agent can be distributed through Microsoft 365 and Teams once the retrieval pattern is working.

The rest of the post is that tree, one branch at a time.

Pattern A: Direct index (classic search, zero agent code)

Start here. Copilot Studio queries hr-policy-index directly through its built-in Knowledge action. No custom agent code runs in the answer path. The sample only owns the index, skillset, and indexing pipeline.

Populate the index (server-side indexer + skillset handles chunking and vectorization):

uv run python scripts/index_knowledge_base_integrated_vectorization.py # Builds hr-policy-index; a client-side alternative exists for dev/test

What you get: very low latency in the sample, roughly 1-2 seconds, no LLM cost in the retrieval path, and native citation cards. When the source documents carry a blob_url or metadata_storage_path, Copilot Studio can render a click-through card straight to the document. For many "where is the policy?" questions, that may be enough.

The honest limitation: Pattern A is still classic search. It does not force synthesis. If Copilot Studio generates an answer from retrieved snippets, it may paraphrase a policy in a way that is close, but not precise enough. For HR policy, that matters. If exact wording matters, that is your sign to step up to Pattern B.

Pattern A2: Copilot Studio meets Foundry IQ (agentic retrieval, no prompt agent)

This is the pattern I would look at when you want better retrieval quality without taking on the overhead of a full prompt agent. In the Copilot Studio new agent experience preview, an agent connects directly to a Foundry IQ knowledge base through Microsoft IQ, with no Foundry prompt agent in between. You reuse the same hr-knowledge-base on top of the same hr-policy-index (one command: python -m src.agents.create_foundry_agent), but retrieval is now agentic: the knowledge base plans sub-queries, retrieves in parallel, reranks, and hands merged results to the agent.

Wiring it takes a few clicks in Copilot Studio (step-by-step on Microsoft Learn):

1. Build → Microsoft IQ → Foundry IQ → Create new connection
2. Choose Microsoft Entra ID Integrated authentication
3. Select hr-knowledge-base
4. Add to agent

A2 is worth the upgrade from A for two reasons. First, you get agentic-retrieval quality without having to build, deploy, or maintain a prompt agent. The knowledge base becomes the reusable asset you improve in Microsoft Foundry, not something you keep reworking inside each Copilot Studio agent. Second, when configured with Microsoft Entra ID Integrated authentication, retrieval can return ACL-trimmed results per user. Each person sees content based on their access.

Foundry IQ knowledge bases can also inherit enterprise-readiness controls such as customer-managed keys, network isolation, and Entra ID. A single knowledge base can also federate across multiple knowledge sources in parallel.

Use A2 when you want stronger hybrid retrieval quality without taking on the overhead of operating a full agent.

Pattern B: Foundry Agent Service with forced grounding

When answers need to be synthesized and grounded, publish a prompt agent to Microsoft Foundry with Foundry Agent Service. In the sample, the agent uses an MCPTool pointing at the knowledge-base endpoint, with tool_choice="required" so the model retrieves policy chunks before answering.

# src/agents/hr_policy_agent.py (excerpt) agent = PromptAgentDefinition( model=model_deployment_name, # e.g. gpt-5-mini instructions=HR_POLICY_INSTRUCTIONS, tools=[mcp_tool], # KB MCP endpoint tool_choice="required", # require retrieval before answering )

Invoke it through the OpenAI client the project hands you:

client = project.get_openai_client() response = client.responses.create( input="How does PTO accrue for a new hire?", extra_body={"agent_reference": {"name": agent_name}}, )

What you get: synthesized answers with grounding and inline [Policy XXXX - Title] citations, all from a single SDK call on a managed runtime. The trade-off: synthesis takes longer. In the sample, answers take roughly 10-14 seconds versus 1-2 seconds for classic search. For policy explanations, that extra time can be worth it because the user gets a composed, grounded answer instead of a list of snippets.

Pattern C: Dual-tool routing for deterministic document locators

Some questions do not need an essay; they just need the right URL, fast. Pattern C lets Copilot Studio route per turn:

• "Where is the PTO policy?" → POST /api/lookup, a deterministic endpoint with no LLM, roughly 1-2 seconds, returning the document URL verbatim in the answer body.
• "How many PTO hours do I accrue?" → hand off to Pattern A or B for a synthesized answer.

POST /api/lookup { "query": "PTO policy" } → 200 OK { "policy_id": "12345", "title": "Types of Leave: Paid Time Off (PTO)", "blob_url": "https://.../12345-pto.pdf" }

Reach for Pattern C when native citations are not enough. For example, use it when you need fast locator responses, the URL printed directly in the answer body, deterministic and auditable output, or support for a source that is not citation-friendly. The endpoint lives at src/backend/main.py:/api/lookup, with its contract in copilot/openapi-lookup-v2.json.

Hosted Agent: the same agent on your own runtime

If you need to own the request loop, custom authentication, side-car services, or infrastructure that stays inside your boundary, run the agent yourself. The Hosted Agent is the self-hosted version of the same idea: a container built on Microsoft Agent Framework with FoundryChatClient. It supports both classic and agentic retrieval through one environment variable:

The context-* modes use Agent Framework’s out-of-the-box RAG context provider. Retrieval runs automatically before each model call with standardized context and citation prompts, so the agent does not have to call a search tool explicitly. That gives the self-hosted path parity with the managed Foundry path across both retrieval types. Copilot Studio can still be the front door. Q3 in the decision tree is really about where the request loop runs, not who greets the user.

Choosing a pattern

A simple way to read the table: start at A, move to A2 when you want agentic retrieval without operating an agent, choose B when each answer needs to be synthesized and grounded in Foundry, add C for high-volume locator traffic, and pick the Hosted Agent when you need the runtime on your own infrastructure. These are not mutually exclusive. A mature agent often routes locator queries to C and content questions to A2 or B.

What's next?

• Try it: clone the sample and follow Steps 1-3 of the walkthrough to stand up Pattern A, provision hr-knowledge-base, connect Copilot Studio, and ask a question in minutes.
• Go agentic: wire the same knowledge base into the Copilot Studio new agent experience via Foundry IQ (Pattern A2) and compare answer quality side by side.
• Learn more: explore agentic retrieval in Azure AI Search, Foundry IQ, and Microsoft Agent Framework.
• Adapt it: swap the HR policy corpus for your own product docs, support runbooks, or internal knowledge source, then compare Pattern A, A2, and B against the same user questions.
• Use the repo-doc map: start with docs/RetrievalPatterns.md for the decision model, docs/CopilotStudioIntegration.md for Pattern A, docs/CopilotStudioHybridExample.md for Pattern C, docs/FoundryAgentArchitecture.md for Pattern B and Hosted Agent, and docs/DataPipelineAndTesting.md for ingestion and validation.

My recommendation: start simple, prove the index works, and move up the stack only when the use case needs it. Some questions need a trusted link. Others need a grounded explanation. A strong architecture supports both without forcing every request through the same path.

 

References

Copilot Studio + Foundry IQ

• Connect to Foundry IQ from an agent
• Foundry IQ FAQ

Foundry IQ / knowledge layer

• What is Foundry IQ?
• Connect a Foundry IQ knowledge base to Foundry Agent Service

Azure AI Search, retrieval

• Agentic retrieval overview
• RAG and generative AI in Azure AI Search
• Classic vs agentic search
• Create a knowledge base
• Create a knowledge source
• Hybrid search
• Semantic ranking
• Quickstart: agentic retrieval
• Tutorial: end-to-end agentic retrieval solution

Microsoft Foundry Agent Service (Pattern B)

• Foundry Agent Service overview

Microsoft Agent Framework (Hosted Agent)

• Microsoft Agent Framework overview
• Hosted MCP tools

Governance

• Azure Well-Architected Framework

Related Microsoft Foundry blog posts

• Foundry IQ is now in Copilot Studio
• Answers You Can Trust: Grounding Enterprise Agents with Foundry IQ
• Foundry IQ: Unlocking ubiquitous knowledge for agents

A critical need, when creating multi-agent applications is to manage token usage in API calls. This is where middleware plays a crucial role. In this post, we will explore what middleware is within the Microsoft Agent Framework, why it is essential for capping token usage, and how to implement it effectively.

What is Middleware in the Microsoft Agent Framework?

Middleware serves as an intermediary layer that processes requests and responses between users and AI agents. It acts as a bridge, allowing developers to intercept, inspect, and modify the data flowing through the system. This capability is vital for implementing additional logic, such as validation, logging, and, importantly, token management.

In the context of the Microsoft Agent Framework, middleware can be utilized to enhance the functionality of AI agents by providing a structured way to handle requests and responses. This not only improves the overall efficiency of the system but also allows for greater control over how the AI interacts with users.

Why Use Middleware to Cap Token Usage?

1. Control Costs

One of the primary reasons to implement middleware for capping token usage is cost management. Many AI services, including those provided by Microsoft, charge based on the number of tokens processed during interactions. By using middleware to monitor and limit token usage, organizations can prevent unexpected spikes in expenses. This is particularly important for businesses that rely heavily on AI for customer service, content generation, or data analysis.

2. Enhance Security

Security is a paramount concern when dealing with AI agents, especially in environments that handle sensitive data. Middleware can validate requests before they reach the agent, ensuring that only legitimate requests are processed. This validation step is crucial for preventing malicious attacks or unintended data exposure, thereby enhancing the overall security posture of the application.

3. Improve Performance

Middleware can also play a significant role in optimizing the performance of AI agents. By intercepting and modifying requests and responses, middleware can streamline the data flow, reducing latency and improving response times. This ensures that the AI agent operates efficiently, providing users with a seamless experience.

4. Custom Logic Implementation

Another advantage of middleware is the ability to implement custom logic tailored to specific business needs. For instance, organizations can modify the input or output of the AI agent based on predefined business rules or user requirements. This flexibility allows for a more personalized interaction, enhancing user satisfaction and engagement.

Types of Middleware in the Microsoft Agent Framework

The Microsoft Agent Framework offers several types of middleware, each serving a unique purpose:

1. Agent Run Middleware

This type of middleware intercepts all agent runs, allowing developers to inspect and modify both input and output. It is particularly useful for implementing global logic that applies to all interactions with the agent.

2. Function Calling Middleware

Function calling middleware intercepts function calls made by the agent, enabling similar inspection and modification capabilities. This is beneficial for managing specific functions that may require additional validation or processing.

3. Streaming Middleware

Designed specifically for handling streaming data, streaming middleware allows for real-time modifications. This is essential for applications that require continuous data flow, such as live chatbots or real-time analytics.

Example of Middleware Implementation

To illustrate how middleware can be implemented to cap token usage, consider the following example in C#:

public class TokenUsageMiddleware
{
    private readonly RequestDelegate _next;
    private const int MAX_TOKENS = 1000; // Define your maximum token limit

    public TokenUsageMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        // Inspect the incoming request
        var tokenCount = CountTokens(context.Request.Body);

        // Cap the token usage
        if (tokenCount > MAX_TOKENS)
        {
            context.Response.StatusCode = StatusCodes.Status400BadRequest;
            await context.Response.WriteAsync("Token limit exceeded.");
            return;
        }

        // Call the next middleware in the pipeline
        await _next(context);
    }

    private int CountTokens(Stream requestBody)
    {
        // Logic to count tokens in the request body
        // This is a placeholder for actual token counting logic
        return 0; // Replace with actual token counting implementation
    }
}

In this example, the TokenUsageMiddleware class intercepts incoming requests, counts the tokens, and checks if the count exceeds a predefined limit. If the limit is exceeded, it returns a 400 Bad Request response, effectively capping token usage.

Additional Real-World Use Cases

Cost Management

Organizations utilizing AI services, such as OpenAI’s GPT models, can implement middleware to monitor and cap token usage. This ensures that they remain within budget and avoid unexpected costs associated with excessive API calls.

Data Validation

Middleware can also be employed to validate user inputs before they reach the AI agent. This prevents invalid or harmful requests from being processed, safeguarding the integrity of the system and enhancing user experience.

Conclusion

Middleware in the Microsoft Agent Framework is a powerful tool for managing interactions with AI agents. By implementing middleware to cap token usage, organizations can effectively control costs, enhance security, and improve performance. This makes middleware an essential component of modern AI applications, enabling businesses to leverage the full potential of AI while maintaining control over their resources.

My blog post application has been updated to cap token usage here.

PyTorch's Inductor compiler automatically groups dependent operations together into single, efficient Triton kernels. This keeps data in faster memory close to the register and cuts down on kernel overhead. In this article, we'll look at an example of fusion, and you'll see exactly how torch.compile transforms your PyTorch operations into optimized GPU code.

When you use PyTorch's compiler, your model runs up to 10x faster, but what's actually happening? Without compilation, the GPU runs a kernel (a function on the GPU) for each torch operation in your code. This causes things to slow down for two reasons: Time is spent moving data in memory, and there's overhead of starting each new kernel. Every time the GPU launches a kernel, it pays an overhead cost, and every intermediate result means writing to and reading from memory. This is where vertical fusion comes in, and why you need to understand how to use it.

To get the most out of this article, you need basic familiarity with PyTorch, and a general understanding of GPU programming concepts.

What is Vertical Fusion?

Think of vertical fusion as a way to link steps, so the output of one goes straight into the next. It's called "vertical" because if you picture the computation graph, these operations stack vertically, with each one dependent on the result of the previous step.

This is the most common fusion pattern in deep learning because neural networks are chains of operations: Normalization, then linear layers, then activation functions, and so on. The big win is eliminating intermediate results. Those temporary tensors never need to be written to or read from global memory. They stay in fast registers where the GPU can reach them more quickly.

Let's dive into an example of vertical fusion, specifically pointwise fusion.

Pointwise fusion example

Pointwise operations are simple math kernels that work on each element: Addition, multiplication, activation functions, and more. Let's look at a pattern you might see in a neural network layer:

import torch

def pointwise_example(x, w, b):
    # Multiple element-wise operations
    tmp = x * w        # multiply
    tmp = tmp + b      # add
    tmp = tmp.sigmoid() # sigmoid activation
    return tmp

Unfused: Three separate kernels

Without fusion, Inductor creates three separate Triton kernels. Don't worry if the Triton syntax looks intimidating. The important part isn't memorizing the syntax, but understanding the pattern. Each kernel loads data, does one operation, and writes the result.

Kernel 1: Multiply

@triton.jit
def mul_kernel(in_ptr0, in_ptr1, out_ptr0, xnumel, XBLOCK: tl.constexpr):
    xoffset = tl.program_id(0) * XBLOCK
    xindex = xoffset + tl.arange(0, XBLOCK)[:]
    xmask = xindex < xnumel
    x0 = xindex
    tmp0 = tl.load(in_ptr0 + x0, xmask)
    tmp1 = tl.load(in_ptr1 + x0, xmask)
    tmp2 = tmp0 * tmp1
    tl.store(out_ptr0 + x0, tmp2, xmask)

For succinctness, I've included just the signatures of the next kernels, because they're nearly identical. See my Git repositoryfor the full source code.

Kernel 2: Add

@triton.jit
def add_kernel(in_ptr0, in_ptr1, out_ptr0, xnumel, XBLOCK: tl.constexpr)

Kernel 3: Sigmoid

@triton.jit
def sigmoid_kernel(in_ptr0, out_ptr0, xnumel, XBLOCK: tl.constexpr)

Across the three kernels, you're performing eight memory operations: Reading inputs twice for multiply, reading multiply's result and the bias for add, reading add's result for sigmoid, and writing all three results. That's a lot of memory traffic.

Fused: One kernel

With fusion, torch.compile creates a single kernel:

Kernel 4: Fused

@triton.jit
def triton_poi_fused_add_mul_sigmoid_0(in_ptr0, in_ptr1, in_ptr2,
                                        out_ptr0, xnumel, XBLOCK: tl.constexpr):
    xoffset = tl.program_id(0) * XBLOCK
    xindex = xoffset + tl.arange(0, XBLOCK)[:]
    xmask = xindex < xnumel
    x0 = xindex

    # Load all inputs once
    tmp0 = tl.load(in_ptr0 + (x0), xmask)
    tmp1 = tl.load(in_ptr1 + (x0), xmask)
    tmp3 = tl.load(in_ptr2 + (x0), xmask)

    # Fused pointwise operations: mul -> add -> sigmoid
    tmp2 = tmp0 * tmp1
    tmp4 = tmp2 + tmp3
    tmp5 = tl.sigmoid(tmp4)

    # Store final result only
    tl.store(out_ptr0 + (x0), tmp5, xmask)

Notice the difference: We load all inputs once, do all three operations in a row, and store only the final result. The intermediate values (tmp2 and tmp4) stay in registers (the fastest memory on the GPU). They never touch the slower global memory.

Benefits of funsion

• Kernel launches: Three reduced to one.
• Intermediate buffers: Two eliminated (multiply result and add result).
• Memory bandwidth: Reading five full tensors and writing three full tensors (eight memory operations) reduced to reading three tensors and writing one (four memory operations). That's a 50% reduction in memory traffic.

Other fusion types

Pointwise fusion is just one type of vertical fusion. Inductor uses other forms of vertical fusion to keep your GPU efficient:

• Reduction fusion: Combines reducing operations like max, mean, or sum, with the operations that happen before and after them. This is critical for operations like batch normalization.
• GEMM + Epilogue fusion: Attaches simple math to the end of heavy matrix calculations. Instead of doing a matrix multiply, writing the result to memory, then reading it back to add bias and apply ReLU, the bias and activation happen right after the multiply in the same kernel.
• Prologue fusion: The opposite of epilogue. Preprocessing happens as data loads. For instance, normalizing input before matrix multiplication can happen even as the data comes in.

In addition to vertical fusion (the most prominent type of fusion), Inductor also uses horizontal fusion.

• Horizontal fusion: Runs multiple independent operations on the same input at once. For example, computing both sin(x) and cos(x) in a single kernel, loading x only once instead of twice.

Get started: See fusion in your own code

Here's a complete example using a reduction pattern.

Step 1: Create a simple reduction example

Create a file called fusion_example.py:

import torch

def reduction_example(x):
    # Pointwise operation followed by reduction
    tmp = x * 2.0
    result = tmp.sum(dim=-1)
    result = result + 1.0
    return result

# Create test input
x = torch.randn(1024, 1024, device='cuda')

compiled_fn = torch.compile(reduction_example)
result_fused = compiled_fn(x)

Step 2: View the generated code

Run your script with the TORCH_LOGS environment variable to see what Inductor generated:

TORCH_LOGS="output_code" python fusion_example.py

This outputs the generated Triton kernels to your terminal. Look for a kernel named something like triton_per_fused_add_mul_sum_0. In this context, per in the kernel name indicates that it's a "per-reduction" kernel. The rest of the name tells you that add, mul, and sum were all fused together.

Conclusion

Fusion is one of the most important optimizations that torch.compile does. By linking dependent operations into single kernels, it cuts down memory traffic and kernel overhead, often the main cause of slow GPU work.

Try accelerating your own code with torch compile. As you've seen, there's no need to change your implementation. Just add a torch compiler decorator, and let the compiler do the work.

Learn more

• PyTorch documentation has complete guides on compilation and optimization strategies.
• Reference my Git repository for the full source code from this article.

The post Why is pytorch compile so fast? appeared first on Red Hat Developer.