alvinashcraft's blurblog

Limit token usage in Microsoft Agent Framework by Jesse Liberty
Wednesday July 29^th, 2026 at 7:14 PM

In the rapidly evolving landscape of artificial intelligence, managing costs associated with API usage is a critical concern for developers. The Microsoft Agent Framework offers a powerful tool in this regard: ChatClientAgentRunOptions. This component allows developers to cap the number of output tokens generated in a single call, ensuring that a single interaction does not exceed budgetary constraints. In this post, we will explore how ChatClientAgentRunOptions functions, its key features, real-world applications, and the challenges developers may face when implementing it.

Overview of ChatClientAgentRunOptions

At its core, ChatClientAgentRunOptions is designed to manage various parameters for chat interactions within the Microsoft Agent Framework. One of its most significant features is the ability to limit the number of tokens generated in a single response through the max_tokens parameter. This capability is essential for controlling costs, as excessive token consumption can lead to budget overruns, especially in applications with high user engagement.

Key Features

Token Management: By specifying a maximum number of tokens that can be generated in a single response, developers can prevent a single interaction from consuming an excessive amount of tokens. This is particularly important in scenarios where the chat agent might otherwise generate verbose or unnecessary responses.
Customizable Options: Beyond token management, ChatClientAgentRunOptions allows developers to fine-tune the behavior of the chat agent through various customizable options. Parameters such as temperature, frequency_penalty, and presence_penalty can be adjusted alongside max_tokens to create a more tailored user experience. For instance, a higher temperature might lead to more creative responses, while penalties can help reduce repetitive outputs.
Integration with Metrics: The Microsoft Agent Framework provides valuable metrics on token usage, enabling developers to monitor both input and output tokens, estimated costs, and latency. This data is crucial for optimizing performance and ensuring that the application remains within budgetary limits.

Real-World Use Case

To illustrate the practical application of ChatClientAgentRunOptions, consider a developer creating a chat agent that provides weather information. By implementing a max_tokens limit, the developer can ensure that the agent does not generate overly verbose responses that could inflate costs. Here’s a simple example of how this might be implemented in Python:

from agent_framework.openai import OpenAIChatClient, OpenAIChatOptions

# Set default options at construction time
agent = OpenAIChatClient().as_agent(
    instructions="You are a helpful assistant",
    default_options={
        "temperature": 0.7,
        "max_tokens": 150  # Limit output tokens to manage costs
    }
)

result = await agent.run("What is the weather like in Amsterdam?")
print(result)

In this example, the developer has set a max_tokens limit of 150. This means that regardless of the complexity of the user’s query, the agent’s response will be capped at 150 tokens, effectively managing costs while still providing valuable information.

Challenges in Token Management

While the capabilities of ChatClientAgentRunOptions are robust, developers must remain vigilant about token usage, particularly in applications with extensive conversation histories. Here are some challenges they may encounter:

Token Overuse: If developers do not actively manage token consumption, the cumulative token count can exceed budget limits. This is especially true in applications where multiple users interact with the agent simultaneously. Developers should implement strategies to monitor and control token usage effectively.
Context Management: Maintaining relevant context in conversations without sending excessive historical data is crucial. Developers are encouraged to trim or limit the stored message history to optimize token usage. This can involve implementing strategies to summarize past interactions or selectively retaining only the most relevant messages.

Conclusion

The ChatClientAgentRunOptions in the Microsoft Agent Framework provides a robust mechanism for managing output tokens, thereby helping developers control costs associated with AI interactions. By leveraging this feature, developers can create efficient and cost-effective chat applications that deliver value to users without exceeding budgetary constraints. As the demand for AI-driven solutions continues to grow, understanding and implementing effective token management strategies will be crucial for developers looking to optimize their applications and maintain financial sustainability.

In summary, the integration of ChatClientAgentRunOptions into your development process can significantly enhance your ability to manage costs while providing a high-quality user experience. By setting appropriate limits on token usage and continuously monitoring performance metrics, developers can navigate the complexities of AI interactions with confidence.

Read the whole story

alvinashcraft

31 minutes ago

reply

Pennsylvania, USA

Uno Platform 6.6: Native AOT, Vulkan Rendering, MCP Auto Registration and More by Uno Platform Team
Wednesday July 29^th, 2026 at 7:14 PM

Uno Platform

&&

Release Uno Platform 6.6

Today, we're announcing Uno Platform 6.6.

This release brings Native Ahead-of-Time (AOT) compilation to Android, iOS, Linux, macOS, and Windows, adds an opt-in Vulkan rendering backend, and makes the Uno Platform MCPs seamless to use.

Uno Platform 6.6 also reduces XAML boilerplate, expands WinUI API coverage, introduces screen-reader support for Skia-rendered applications, and improves multilingual text input and rendering with complete IME composition and automatic font fallback.

Uno Platform 6.6 is available today. See the documentation for the different ways to get started or upgrade an existing project.

Native AOT

Native AOT Across Five Platforms

Starting with v6.6, Uno Platform applications can be published with Native AOT, delivering near-instant startup and smaller deployment packages across mobile and desktop.

Using Uno.Chefs on .NET 10, we measured startup improvements of up to 60%, reflecting the gains developers can expect from a full-scale application rather than an idealized minimal sample. The result is a faster first launch and a leaner application footprint.

AOT is best suited to applications where startup performance is a priority. JIT publishing remains fully supported, giving teams the flexibility to choose the publishing model that best fits their application and development workflow.

Platform	Default Runtime	Native AOT	Faster By
Android	0.895s	0.348s	61%
iOS	0.940s	0.742s	21%
Linux	0.870s	0.350s	60%
macOS	1.347s	0.555s	59%
Windows	1.605s	0.824s	49%

The complete benchmark table, methodology, and setup instructions are available in the Native AOT documentation. You can also read about how we contributed AOT improvements to .NET for Android.

MCP Setup

Seamless MCP Setup via Auto-Registration

Uno Platform has provided two Model Context Protocol servers.

MCP	What It Gives Your Agent
App MCP	Access to the behavior of a running application, allowing the agent to verify what the application actually does: AI 'eyes and hands'
Docs MCP	Access to latest Uno Platform documentation, grounding agentic responses in best practices, APIs, and guidance available today

In Uno Platform 6.6, these MCPs can register themselves with your IDE. This reduces the manual setup required before an AI agent can consult Uno Platform documentation or interact with a running application, so you spend less time connecting tools and more time using them.

Agent	Auto Registration Support
Claude Code	Yes
Copilot (VS Code)	Yes
Copilot (Visual Studio)	Yes
Copilot (CLI)	Yes
Cursor	Yes
Kiro	Yes
Gemini CLI	Yes
Codex CLI	Yes
Gemini Antigravity	No
Claude Desktop	No
Windsurf	No
Junie Rider	No
JetBrains Air	No

For the most up to date documentation on MCP setup and troubleshooting, see the MCP documentation.

Vulkan

Opt-In Vulkan Rendering for Better Performance

Uno Platform 6.6 introduces an opt-in Vulkan rendering backend for Windows, Linux, and Android.

Vulkan can reduce the rendering cost of each frame, with the largest benefits expected on Android devices and desktop systems with modern GPUs, with performance improvements of up to 50% observed in some cases. This can translate into smoother animations and more responsive graphics-heavy experiences while preserving the same rendering output.

Vulkan is optional, so teams can enable it where rendering performance matters most and compare the results with their current backend. Apple platforms continue to use Metal.

Platform	Vulkan Backend	Default Backend
Windows	Supported (opt-in)	Skia, using OpenGL
Linux	Supported (opt-in)	Skia, using OpenGL
Android	Supported (opt-in)	Skia, using OpenGL
iOS	Not available	Skia, using Metal
macOS	Not available	Skia, using Metal
WebAssembly	Not available	Skia, using WebGL

XAML

Less Boilerplate in XAML Projects

Uno Platform 6.6 introduces two improvements that make XAML files and project structures more concise. The default WinUI presentation namespace and the x: namespace are now implicitly available on Uno Platform targets.

Before

<Page
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    x:Class="MyApp.MainPage">
  <StackPanel>
    <Button Content="Hello" />
  </StackPanel>
</Page>

After (6.6)

<Page x:Class="MyApp.MainPage">
  <StackPanel>
    <Button Content="Hello" />
  </StackPanel>
</Page>

This change removes repeated namespace declarations from individual XAML files, making pages easier to scan without affecting how their controls behave. Developers can define a namespace once in a central file, such as GlobalNamespaces.xaml, and make it available across every XAML file in the project.

Applications and libraries can also expose their own namespaces implicitly. The documentation covers the registration and namespace resolution details.

Automated Code-Behind Source Code Generation

Uno Platform 6.6 introduces automatic code-behind generation for XAML pages. Any page that declares x:Class now gets its generated class produced automatically at build time, no default .xaml.cs file required.

That means a XAML-only page can now ship as a single file: MainPage.xaml. And that's it. Uno Platform handles the rest.

Developers who need constructor logic, event handlers, lifecycle overrides, or other custom behavior can still add a manual code-behind file at any time, giving full flexibility without any required boilerplate.

WinUI Coverage

Expanded WinUI API Coverage

Uno Platform 6.6 expands the set of WinUI and WinRT capabilities available across non-Windows targets. This allows developers to share more application code and XAML across Windows, WebAssembly, Android, iOS, macOS, and Linux without replacing familiar APIs with platform-specific alternatives.

Area	What Is New	What It Enables
Layout and visuals	PlaneProjection, Matrix3DProjection, Geometry.Transform GetAlphaMask()	More advanced transforms, visual effects, and composition
Text	Spell checking, text highlighting, additional text-trimming support	More complete editors, forms, and document experiences
Menus and interaction	Expanded flyout support, context menus, scroll anchoring, XAML event-trigger actions	Richer desktop interactions using familiar WinUI patterns
Drag, drop, and clipboard	Broader clipboard support and Windows desktop drag and drop for virtual and in-memory files	Better productivity, file-management, and content workflows
WebView2	Resource interception, flicker/z-ordering/message lifecycle improvements	More reliable hybrid application experiences
Theming and backdrops	Element-level RequestedTheme, Mica and Acrylic window backdrops on macOS	More flexible themes and platform-appropriate desktop visuals
Media	Animated WebP playback	Richer image and animation experiences

The complete API list and platform-specific notes are available in the release documentation.

Accessibility

Screen-Reader Support for Skia-Rendered Apps

Uno Platform 6.6 allows Skia-rendered applications to expose their interfaces to screen readers on Windows, macOS, and WebAssembly, with further target platform support to come.

Developers can continue defining the semantic meaning of their controls through Uno Platform's accessibility abstractions. Uno Platform then makes that information available to the accessibility services provided by each supported environment.

Platform	Screen-Reader Support in 6.6
Windows	Available
macOS	Available
WebAssembly	Available
Android	In progress
iOS	In progress
Linux	Planned

The result is not only better access for screen-reader users and compliance to standards, but also a more inspectable and testable interface for development teams.

Multilingual

Complete Multilingual Text Input and Rendering

Uno Platform 6.6 delivers two major improvements for applications used across languages and writing systems: complete Input Method Editor composition and automatic font fallback.

IME Composition

Input Method Editors are used to enter languages and writing systems that cannot be represented directly by individual keyboard keys, including Chinese, Japanese, Korean, and Vietnamese.

Uno Platform 6.6 adds complete IME composition support across Windows, WebAssembly, Android, iOS, macOS, and Linux. Users can now compose, review, and confirm characters using the input methods already built into their operating system. The feature is available out of the box, with no additional configuration required.

Automatic Font Fallback

A single font does not always include a visual glyph for every character. When a glyph is unavailable, the character may appear as an empty square, commonly known as a tofu box.

Uno Platform can now select a suitable fallback font for the characters that need one. This allows Latin, CJK, Arabic, Georgian, and other supported scripts to render correctly within the same sentence. Developers no longer need to manually choose a single font that contains every character used by their interface.

SkiaSharp

SkiaSharp 4.x Is Available as an Opt-In

Since the Uno Platform 6.6 release branch was created, Uno Platform has joined the .NET team as a co-maintainer of SkiaSharp and helped ship SkiaSharp 4.0.

Uno Platform 6.6 defaults to SkiaSharp 3.x, but you can opt into SkiaSharp 4.x when it suits your project. For applications that use SkiaSharp directly, the 4.x line brings a newer underlying Skia engine alongside capabilities such as variable-font axes, color-font palettes, animated WebP encoding, and upstream rendering, codec, performance, and security improvements.

SkiaSharp 4.x compatibility and opt-in instructions are available in the documentation.

Watch

See the full story: SkiaSharp 4.0 launch session with the Microsoft and Uno Platform teams, covering what's new, real-world examples, and creative demos showing what SkiaSharp can do beyond the basics.

Contributors

Thank You to Our Contributors

Uno Platform 6.6 was made possible by 26 contributors from across the Uno Platform team and the wider community.

Thank you to everyone who contributed code, tested preview builds, reviewed changes, filed reports, or helped us identify where the platform could work better. Community contributions from developers including tmds and clairernovotny helped improve this release.

Part of Uno Platform 6.6 belongs to everyone who tried an early build and took the time to tell us what they found.

Upgrade

Upgrade to Uno Platform 6.6

To upgrade an existing project, update your Uno Platform IDE extension and bump the Uno SDK version in your project's global.json file:

global.json

{
  "msbuild-sdks": {
    "Uno.Sdk": "6.6.23"
  }
}

If you're upgrading from an earlier release review the migration guide.

New to Uno Platform? The easiest way to get started is with Uno Platform Studio, directly in your browser. Prompt your own app idea or continue building the Brewhouse application we’ve already started for you. No setup required.

&

The post Uno Platform 6.6: Native AOT, Vulkan Rendering, MCP Auto Registration and More appeared first on Uno Platform.

Read the whole story

alvinashcraft

31 minutes ago

reply

Pennsylvania, USA

Fast Web Services with Marten and Polecat by jeremydmiller
Wednesday July 29^th, 2026 at 7:14 PM

The Shade Tree Developer

In many .NET systems, writing a web service that returns query results means some combination of:

Query data from EF Core — which is going to do who knows what to build up SQL, execute that, then spend some time materializing the raw database results into .NET objects
Since we’ve all been taught for years that it’s harmful to expose our internal entity shapes to the outside world, maybe you’re running the results through some kind of object to object mapping to a different DTO shape
Finally, after all the database querying and object mapping, you’ll finally use a JSON serializer to write results to the HTTP response stream

Whew. That’s a non-trivial amount of your time (or AI tokens) and a significant amount of runtime overhead with all the transformations and thrashing your memory with all the object allocations involved.

Now let’s talk about some capabilities in Marten and Polecat to sidestep the mass majority of that overhead in some cases — but first, I do need to say that if you’re using Event Sourcing, the persisted data in a Marten or Polecat database for query models is purpose built for clients as it is. No extra mapping necessary. In a way, the “AutoMapper” activity happens directly in projections for a system using Event Sourcing.

If you are building HTTP services on top of Marten or Polecat, both of these tools have a “JSON Streaming” feature that can be used to build very fast web services by writing the raw JSON stored in PostgreSQL or SQL Server directly to the HTTP response for the most efficient possible HTTP web services in the read side of a CQRS architecture.

Core team member Anne Erdtsieck just made some a bunch of extensions to Marten and Polecat‘s ability to stream the raw, persisted JSON data stored in the database straight to HTTP responses, and that makes now a good time to show off what we have.

For Minimal API endpoints (and for frameworks like Wolverine.Http that dispatch any IResult return value), Marten.AspNetCore (Polecat.AspNetCore has similar support) ships seven typed result wrappers that carry the streaming behavior above as endpoint return values while also contributing correct OpenAPI metadata:

Type	Source	Response shape	404 on miss?
`StreamOne<T>`	`IQueryable<T>` — regular Marten document query	Single `T`	yes
`StreamMany<T>`	`IQueryable<T>` — regular Marten document query	JSON array `T[]`	no (empty array = 200)
`StreamAggregate<T>`	`IDocumentSession` + stream id — event-sourced	Single `T`	yes
`StreamPaged<T>`	`IQueryable<T>` — regular Marten document query	Paged JSON envelope	no (empty page = 200)
`StreamPagedByCursor<T>`	`IQueryable<T>` (with `OrderBy`/`ThenBy`)		no (empty array = 200)
`StreamEventState`	`IQuerySession` + stream id — event stream	Single `StreamStateResponse`	yes
`StreamEvents`	`IQuerySession` + stream id — event stream	JSON array `EventResponse[]`	yes (configurable)

Each type implements both IResult (so ASP.NET Minimal API dispatches it via ExecuteAsync) and IEndpointMetadataProvider (so Swashbuckle, NSwag, and the built-in OpenAPI generator see the right response shape), while delegating the actual body write to WriteSingle/WriteArray/WriteLatest/WriteStreamState/WriteEvents. Returning one from an endpoint is a concise, typed alternative to writing the HTTP handshake manually.

`StreamOne<T>` — single document with 404 on miss

			
app.MapGet("/issues/{id:guid}",
    (Guid id, IQuerySession session) =>
        new StreamOne<Issue>(session.Query<Issue>().Where(x => x.Id == id)));

Returns 200 application/json with the document JSON on a hit, 404 on a miss. Content-Length and Content-Type are set automatically, matching the behavior of WriteSingle<T>.

`StreamMany<T>` — JSON array

			
app.MapGet("/issues/open",
    (IQuerySession session) =>
        new StreamMany<Issue>(session.Query<Issue>().Where(x => x.Open)));

Returns 200 application/json with a JSON array body. An empty result set yields [], not a 404 — matching the behavior of WriteArray<T>.

`StreamPaged<T>` — paged JSON envelope (single round trip)

			
app.MapGet("/issues/paged/{pageNumber:int}/{pageSize:int}",
    (int pageNumber, int pageSize, IQuerySession session) =>
        new StreamPaged<Issue>(session.Query<Issue>().OrderBy(x => x.Description), pageNumber, pageSize));

Returns 200 application/json with a single JSON envelope combining paging metadata and the matching documents for that page:

			
{"pageNumber":3,"pageSize":25,"totalItemCount":1207,"pageCount":49,"hasNextPage":true,"hasPreviousPage":true,"items":[...]}

pageNumber is 1-based. totalItemCount and pageCount are computed from a count(*) OVER() window function added to the same SQL query that fetches the page, so the whole response — count and documents both — comes from a single database round trip. Documents inside items are streamed as raw, already-persisted JSON, without a deserialize/serialize step. An empty page still returns 200 with totalItemCount: 0, pageCount: 0, and an empty items array — never a 404.

Internally, StreamPaged<T> delegates to the IQueryable<T>.StreamPagedJsonArray() extension method described in the Paging docs, which can also be used directly (e.g. from an MVC controller action) instead of through the IResult wrapper.

`StreamAggregate<T>` — event-sourced aggregate (latest)

			
app.MapGet("/orders/{id:guid}",
    (Guid id, IDocumentSession session) =>
        new StreamAggregate<Order>(session, id));

Returns 200 application/json with the JSON of the latest projected aggregate state, or 404 if no stream exists. A constructor overload accepts string ids for stores configured with string-keyed streams.

`StreamEventState` — event stream metadata

Writes the high level metadata of a single event stream — Marten’s StreamState — as JSON, or 404 when the stream does not exist:

			
app.MapGet("/minimal/order/{id:guid}/state",
    (Guid id, IQuerySession session)
        => new StreamEventState(session, id));

A constructor overload accepts a string stream key for stores configured with string-keyed streams.

The response body is a StreamStateResponse, not StreamState itself. StreamState.AggregateType is a System.Type, and System.Text.Json refuses to serialize those outright (Serialization and deserialization of 'System.Type' instances is not supported), so the aggregate type is projected down to its simple name in AggregateTypeName:

			
{
  "id": "0198e1b4-5b1c-7a1e-9a3f-2f2f5b6c7d8e",
  "key": null,
  "version": 2,
  "aggregateTypeName": "Order",
  "lastTimestamp": "2026-07-26T09:41:02.113Z",
  "created": "2026-07-26T09:41:02.098Z",
  "isArchived": false
}

		

`StreamEvents` — raw events of a stream 9.20

Writes the raw events of a single event stream as a JSON array:

			
app.MapGet("/minimal/order/{id:guid}/events",
    (Guid id, IQuerySession session)
        => new StreamEvents(session, id));

StreamEvents carries the same optional version, timestamp, and fromVersion filters as FetchStreamAsync(), and there is a string stream key overload as well.

Elements are EventResponse, not IEvent itself — IEvent.EventType is a System.Type and hits the same System.Text.Json wall as above. Use eventTypeName, Marten’s stable event type alias, to discriminate event types on the client. The assembly qualified .NET type name (DotNetTypeName) is deliberately left off the wire:

			
[
  {
    "id": "0198e1b4-5b1c-7a1e-9a3f-2f2f5b6c7d8e",
    "version": 1,
    "sequence": 41,
    "streamId": "0198e1b4-5b1c-7a1e-9a3f-2f2f5b6c7d8e",
    "streamKey": null,
    "eventTypeName": "order_placed",
    "timestamp": "2026-07-26T09:41:02.098Z",
    "tenantId": "*DEFAULT*",
    "isArchived": false,
    "causationId": null,
    "correlationId": null,
    "headers": null,
    "data": { "description": "Widget", "amount": 99.95 }
  }
]

		

Empty streams: 404 or an empty array?

FetchStream yields an empty list both for a stream that does not exist and for a filter that excludes every event, and the two cannot be told apart. StreamEvents therefore exposes an OnEmptyStatus that defaults to 404, matching the other single-resource results. Set it to 200 when running off the end of a stream is expected rather than exceptional — paging forward with fromVersion, for example:

			
// Paging forward through a stream: running off the end is expected, not a 404
app.MapGet("/minimal/order/{id:guid}/events/from/{fromVersion:long}",
    (Guid id, long fromVersion, IQuerySession session)
        => new StreamEvents(session, id, fromVersion: fromVersion)
        {
            OnEmptyStatus = StatusCodes.Status200OK
        });

		

Read the whole story

alvinashcraft

32 minutes ago

reply

Pennsylvania, USA

Tool search: Finding the right tool at the right time by Viswajeeet Balaji, Saket Sathe
Wednesday July 29^th, 2026 at 7:13 PM

Command Line

Every tool you give an agent is both a capability and a distraction. Five tools make an agent feel capable. Fifty tools make it feel prepared. A hundred tools can make every turn start with thousands of tokens of names, descriptions, JSON schemas, argument definitions, and nested parameters before you’ve asked anything useful. The agent looks more powerful, but first it has to read a menu it may not need.

This is one of the tensions Toolboxes in Microsoft Foundry is designed to solve at enterprise scale. A single toolbox can front Microsoft IQ, Work IQ, OpenAPI tools, A2A integrations, remote MCP servers, and several native Azure capabilities. Agent builders should be able to scale heterogeneous tool catalogs without rebuilding integrations or loading every tool on every turn. The old line about great power going hand-in-hand with great responsibility applies here, but the responsibility lives at the context layer: if the platform makes it easy to connect everything, it also needs a way to keep the model focused on what matters now.

Tool search capability in Toolbox emerged as we worked backwards from customer experience. Large tool catalogs were becoming too expensive to send to the model on every turn, and the model didn’t need most of them for most tasks. Our initial experiment of deferring all the tools and letting the model search for tools based on user query did what we hoped: it made tool-using agents cheaper, made the prompt smaller, and kept the system prompt stable enough to work well with prompt caching.

Then the real story emerged. We thought we were solving a token-cost problem; we were also building a search product. At small scale, a catalog can look like schema management: register the tool, validate the JSON, expose it to the model, dispatch the call. At larger scale, tool names and descriptions become ranking features. That’s the shift from tool-maxxing to tool relevance-maxxing.

The default agent tax

The problem isn’t just cost, though cost is the easiest part to measure. A full manifest also fills the context window with definitions unrelated to the current task. The model has to choose from an overcrowded menu, and the prompt prefix becomes larger and more fragile.

Prompt caching is enabled by default in Azure OpenAI and is the recommended behavior, so our baseline had to include it. But caching isn’t the same as not loading. Cached tokens are roughly 90% cheaper than regular input tokens, not free, and cached context still competes for the model’s attention. The obvious move was to stop loading everything upfront.

Two tools instead of a hundred

Tool search changes the initial contract between the toolbox and the model. Instead of exposing every toolbox tool in the first tools/list, Foundry can expose two meta-tools: tool_search(query, limit) and call_tool(name, arguments). The model describes the capability it needs, Foundry searches the toolbox, and the model receives a small set of matching tool definitions before calling the chosen tool with name and arguments.

The rest of the catalog stays hidden from the initial tool list. That’s why the second proxy exists: if a tool wasn’t registered in the original tools/list, many runtimes will guard against the model calling it directly as an unknown tool. call_tool gives the framework a registered, policy-aware dispatch path.

An architectural diagram showing how tool search surfaces two tools rather than 100 tools

That sounds almost too small to be an architecture, which is partly why we liked it. It works inside today’s tool-calling contract and doesn’t depend on a model-specific feature. Strategically placing this at the Toolbox layer is critical because remote MCP servers, OpenAPI tools, A2A integrations, native Azure tools, and other entries can be discovered through one mechanism. Foundry indexes the tool name, description, argument names, and argument descriptions up to three levels of nesting.

That minimal shape was deliberate. We didn’t want builders to tune a retrieval system before they had shipped an agent. The default should be sane enough to try immediately: attach a toolbox, enable search, and let the platform choose the initial indexing and ranking behavior. The surface also stays model-agnostic: no special model family, new MCP primitive, or shared ranking semantics across every provider. Builders only need to tell the agent to call tool_search before concluding a capability is missing, and they can then steer the shortlist size with limit: five results by default, more for ambiguous workflows, fewer for narrow ones.

The savings were real

We evaluated tool search on ToolRet, a large-scale open benchmark with more than 44,000 tools and 7,000 queries. The goal was to measure how token savings change as the catalog grows, so we ran an ablation that increased the number of tools available to the model and compared token consumption with and without tool search. The full tradeoff is shown in Figure 2 below.

A chart showing tokens saved using tool search compared to baseline

The savings scaled with toolbox size. With 50 tools in the toolbox, tool search reduced token use by more than 60%. With 1,000 tools, the savings rose above 97%. In this baseline, every tool was provided to the model upfront, and the model chose from the full catalog.

Retrieval quality was the real test

Retrieval quality was the harder question. We measured it with Recall@10 and compared tool search against BM25s and BGE-reranker-v2-gemma, as shown in Figure 3. For this run, we used only the user query and left out the benchmark’s instruction string. Generating that instruction at runtime would require another LLM call, adding both cost and latency, so the test reflected the cheaper path we would want in production. Tool search uses an enhanced sparse-retrieval pipeline with lexical similarity matching, and its Recall@10 was comparable to GPU-based reranking without depending on expensive cross-encoder reranking at serving time.

	Web	Code	Customized
Tool search	45.99%	39.56%	41.36%
BM25s	24.62%	28.23%	32.39%
BGE-reranker-v2-gemma	45.94%	38.23%	49.43%

Figure 3: Comparing Recall@10 across various methods. BM25s, BGE-reranker-v2-gemma results are from [1].

The table shows tool search improvement over BM25s in all three categories: web, code, and customized. It is nearly tied with BGE-reranker-v2-gemma on web, slightly ahead on code, and behind on customized. Clearly, in two of the three categories, tool search is competitive with the GPU-based reranker, without the GPU cost.

These results show that, in addition to cost optimization, tool search can preserve strong retrieval quality while shrinking the tool context the model has to carry. As tool catalogs grow, the advantage comes from making the right tools discoverable at the right moment without paying the full cost of exposing everything upfront.

Tuning the search space

Benchmark testing revealed that tool search failed when tool descriptions were uneven, sometimes capturing implementation detail instead of user intent vocabulary. Some descriptions were too generic: “get,” “create,” “manage,” “REST API.” The terms had to be reflective of actual user queries.

Toolbox allows developers to configure an optional search-only text field, additional_search_text, for every tool. This additional text is indexed and helps discover a particular tool with higher accuracy. This additional text isn’t visible to models in MCP responses, thus helping you keep all the token savings. No changes are made to the original tool schema of the source MCP server. The returned schema stays clean while the search index learns aliases, domain terms, internal names, and user vocabulary.

For example, a database tool called execute_query might need to be found when a user says “analytics,” “dashboard,” “SQL,” “reporting,” or “warehouse.” A description like “runs a query against the configured database” may be accurate, but it isn’t very searchable. Search-only text can add terms users and models reach for: “analytics query, dashboard data, SQL report, warehouse lookup, inspect tables.” With tuned metadata, retrieval hit rate improved by about 56%, and end-to-end accuracy improved by about 55%, recovering to within about 4% of the full-catalog baseline.

This is where the work became more interesting than simply “saving tokens.” Tool search turned tool curation into an information-retrieval discipline. Adding a tool now raises different questions: what words will a user use, is the description specific enough to beat nearby tools, and is the tool too important to rely on retrieval?

Search is for the long tail

Most tool use has a Pareto shape. A small fraction of tools handles most tasks, but the long tail still matters because rarely used tools are often exactly the ones you need in high-stakes moments: rotate a credential, recover a failed deployment, apply a compliance exception, inspect an audit trail. Search is a good default for that long tail. It is not a good default for tools the model constantly needs.

Several tools are part of the agent’s core contract: policy tools, frequently used data access tools, or capabilities the model should never have to rediscover. Toolbox auto-pins frequently used tools based on usage at a per user level. Auto-pinned tools will be visible in tools/list call after a warmup period, with stale entries aging out as usage changes. Developers can also manually pin tools on top of this. Deterministic pinning keeps the prompt prefix stable, which preserves prompt-cache behavior.

When we would use tool search

If your toolbox has more than 10–15 tools, different tasks need different subsets, or one agent serves many workflows, tool search is worth testing. It’s most useful when the manifest is becoming a material part of cost, when the catalog changes often, or when no fixed tool subset is right.

Tool search is less compelling for tiny catalogs, agents that almost always use the same few tools, or catalogs whose descriptions are too vague to improve. Smaller is not better if the right capability disappears. Tool search changes the agent from “choose from everything” to “ask for a shortlist.” The shortlist has to be good.

Try it

Tool search is in preview for Toolboxes in Foundry. Start with the Microsoft Learn tool search docs, enable toolbox_search on a versioned toolbox, and test before promoting. Then inspect the misses. The first useful tuning pass probably won’t be algorithmic. It will be editorial: improve descriptions, add additional_search_text for domain vocabulary, and pin the tools that are part of the agent’s core contract.

  
import os 
from azure.identity import DefaultAzureCredential 
from azure.ai.projects import AIProjectClient 
from azure.ai.projects.models import MCPTool, ToolSearchToolboxTool 
  
client = AIProjectClient( 
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], 
    credential=DefaultAzureCredential(), 
) 
  
# ToolboxSearchToolType() enables tool search — other tools in the toolbox are discovered on 
# demand through tool_search instead of being listed up front. Add as many MCP servers as you need; 
# tool search keeps the agent's initial tool surface small regardless of toolbox size. 
toolbox_version = client.toolboxes.create_version( 
    name="my-toolbox", 
    description="Large toolbox with tool search enabled", 
    tools=[ ToolboxSearchToolboxTool(), 
{ 
    "type": "mcp", 
    "server_label": "analytics", 
    "server_url": "https://db-mcp.internal/sse", 
    "tool_configs": { 
        "execute_query": { 
            "pin": True, 
            "additional_search_text": "SQL database analytics reporting dashboard queries", 
        }, 
        "list_tables": { 
            "additional_search_text": "schema columns metadata table structure discover", 
        }, 
    }, 
}], 
) 
print(f"Created toolbox `{toolbox_version.name}` (version {toolbox_version.version})") 
  

For more detailed steps on integration with the agent framework of your choice, click here.

Enable tool search with one click in the Foundry Portal:

The post Tool search: Finding the right tool at the right time appeared first on Command Line.

Read the whole story

alvinashcraft

32 minutes ago

reply

Pennsylvania, USA

In AI-Native Delivery, Architecture Has to Say Where to Stop by Mirco Hering
Wednesday July 29^th, 2026 at 7:12 PM

Not A Factory Anymore

When you heat metal and cool it slowly, its internal stresses are relieved. The internal structure settles into a more stable state. This is annealing, and I think it is a useful way to think about software architecture when AI agents are doing part of the delivery work. Hot and cold are not just descriptions. […]

Read the whole story

alvinashcraft

33 minutes ago

reply

Pennsylvania, USA

Version 0.1.7 by afourney
Wednesday July 29^th, 2026 at 7:12 PM

Release notes from markitdown

What's Changed

fix: avoid O(n^2) value lookups in PPTX chart conversion by @kz-000 in #2227
Fix invalid LaTeX macros for mu, nu, tau, and down-arrow in equation conversion by @ksprajapatii in #2228
Fix typos and formatting in comments, docstrings, and markdown by @chienyuanchang in #2223
fix: handle PPTX SVG images without a rasterized fallback by @guoyu-wang in #2233
Fix omml template bugs. by @afourney in #2257

New Contributors

@kz-000 made their first contribution in #2227
@ksprajapatii made their first contribution in #2228
@guoyu-wang made their first contribution in #2233

Full Changelog: v0.1.6...v0.1.7

Read the whole story

alvinashcraft

33 minutes ago

reply

Pennsylvania, USA

Limit token usage in Microsoft Agent Framework by Jesse Liberty Wednesday July 29th, 2026 at 7:14 PM

Overview of ChatClientAgentRunOptions

Key Features

Real-World Use Case

Challenges in Token Management

Conclusion

Uno Platform 6.6: Native AOT, Vulkan Rendering, MCP Auto Registration and More by Uno Platform Team Wednesday July 29th, 2026 at 7:14 PM

Native AOT Across Five Platforms

Seamless MCP Setup via Auto-Registration

Opt-In Vulkan Rendering for Better Performance

Less Boilerplate in XAML Projects

Automated Code-Behind Source Code Generation

Expanded WinUI API Coverage

Screen-Reader Support for Skia-Rendered Apps

Complete Multilingual Text Input and Rendering

IME Composition

Automatic Font Fallback

SkiaSharp 4.x Is Available as an Opt-In

Thank You to Our Contributors

Upgrade to Uno Platform 6.6

Fast Web Services with Marten and Polecat by jeremydmiller Wednesday July 29th, 2026 at 7:14 PM

StreamOne<T> — single document with 404 on miss

StreamMany<T> — JSON array

StreamPaged<T> — paged JSON envelope (single round trip)

StreamAggregate<T> — event-sourced aggregate (latest)

StreamEventState — event stream metadata

StreamEvents — raw events of a stream 9.20

Empty streams: 404 or an empty array?

Tool search: Finding the right tool at the right time by Viswajeeet Balaji, Saket Sathe Wednesday July 29th, 2026 at 7:13 PM

The default agent tax

Two tools instead of a hundred

The savings were real

Retrieval quality was the real test

Tuning the search space

Search is for the long tail

When we would use tool search

Try it

In AI-Native Delivery, Architecture Has to Say Where to Stop by Mirco Hering Wednesday July 29th, 2026 at 7:12 PM

Version 0.1.7 by afourney Wednesday July 29th, 2026 at 7:12 PM

What's Changed

New Contributors

Limit token usage in Microsoft Agent Framework by Jesse Liberty
Wednesday July 29^th, 2026 at 7:14 PM

Uno Platform 6.6: Native AOT, Vulkan Rendering, MCP Auto Registration and More by Uno Platform Team
Wednesday July 29^th, 2026 at 7:14 PM

Fast Web Services with Marten and Polecat by jeremydmiller
Wednesday July 29^th, 2026 at 7:14 PM

`StreamOne<T>` — single document with 404 on miss

`StreamMany<T>` — JSON array

`StreamPaged<T>` — paged JSON envelope (single round trip)

`StreamAggregate<T>` — event-sourced aggregate (latest)

`StreamEventState` — event stream metadata

`StreamEvents` — raw events of a stream 9.20

Tool search: Finding the right tool at the right time by Viswajeeet Balaji, Saket Sathe
Wednesday July 29^th, 2026 at 7:13 PM

In AI-Native Delivery, Architecture Has to Say Where to Stop by Mirco Hering
Wednesday July 29^th, 2026 at 7:12 PM

Version 0.1.7 by afourney
Wednesday July 29^th, 2026 at 7:12 PM