Enterprise content is no longer just something people read. AI apps and agents are only as useful as the information they can understand, yet much of the world’s enterprise knowledge is locked in documents, forms, tables, images, audio, and video. The latest Azure Content Understanding updates help developers turn that content into structured, grounded data with less custom processing. This release brings broader support for the GPT-5 model series, lower token usage, improved confidence scoring, new synchronous APIs for Read and Layout, advanced contextualization for higher-quality extraction, new tax-focused prebuilt analyzers, semantic chunking for retrieval workflows, and agentic document reasoning for more complex extraction scenarios.

Specifically, we’re announcing two updates to Azure Content Understanding in Foundry Tools:

• A refreshed CU 1.0 API, now generally available for production workloads. CU 1.0 adds broader GPT-5 series support, lower token use, and improved grounding and confidence scoring.
• A new CU 2.0 public preview for developers exploring next-generation document understanding. CU 2.0 adds synchronous Read and Layout APIs, advanced contextualization, semantic chunking, improved classification, new prebuilt analyzers, and agentic document reasoning.

Together, these updates make Content Understanding (CU) more efficient for production workloads today while expanding the range of document automation, retrieval, and reasoning scenarios developers can build tomorrow.

CU 1.0: Better economics and reliability for production workloads

We’ve made significant enhancements to the CU 1.0 GA API (2025-11-01), allowing customers to take advantage of broader GPT-5 series support, improved grounding efficiency, and a refreshed confidence model in their existing production workflows. Each of the three improvements below targets a different part of that production path: the model you run, the tokens it costs, and the confidence you can place in the result.

Expanded support for GPT-5.5 and lower GPT-5 series model support

Content Understanding analyzers can now use the GPT-5 model series, including GPT-5.5, GPT-5.4, GPT-5.3, GPT-5.2, GPT-5.1, and GPT-5 series models, across standard, mini, and nano variants.

This gives developers more flexibility to choose the right model deployment for their workload. Some scenarios prioritize maximum accuracy. Others prioritize latency or cost. By expanding GPT-5 series support, CU enables customers to evaluate these tradeoffs on their own data while continuing to use the same analyzer model and API patterns. Customers can use existing PTU commitments.

Improved grounding efficiency

The GA refresh also improves grounding efficiency by merging extraction and grounding more effectively in the processing flow. In internal evaluations, this reduced average inference token usage by up to 28 percent for GPT-4.1 and GPT-5.2, while also improving average accuracy by up to 3 percent.

For customers building high-volume extraction workflows, this matters in two ways. First, lower token usage can reduce the cost of running LLM-backed extraction. Second, better grounding efficiency helps preserve traceability to the source content without requiring developers to add extra post-processing logic.

Refreshed confidence model

The GA refresh includes a refreshed confidence model. In internal evaluations, accuracy measured by AUROC improved by up to 14 percent for GPT-4.1 and GPT-5.2.

Confidence scores are a critical part of production automation. They help applications decide when to move straight through, when to route to human review, and when to apply additional validation. A stronger confidence model makes it easier for teams to build automated pipelines that are both efficient and auditable.

For guidance on how to choose the right model for specific tasks and more details on our quality benchmarks see Azure Content Understanding GPT-5 Series Guide: Model Selection, Grounding Improvements, and Confidence Enhancements.

CU 2.0 preview: New building blocks for AI apps and agents

In addition to the enhancements available in the refreshed GA API, we are introducing the new CU 2.0 Preview (2026-06-01-preview).

CU 2.0 preview advances two developer priorities:

1. Improving the quality and efficiency of content pipelines, and;
2. Enabling new low-latency and reasoning-intensive scenarios.

The capabilities below build on the same GPT-5 series foundation as the refreshed GA API.

GPT-5.5 and lower models supported in preview

The CU 2.0 public preview also supports the GPT-5 model series. This means customers can test new preview capabilities using the same model series direction as the refreshed GA path.

This is important for migration planning. Customers can evaluate accuracy, latency, and cost on representative data before deciding which model deployment and API version best fit their production needs.

Improving quality and efficiency for production workloads

The first goal of the new CU 2.0 preview version is to improve the work teams already run in production: raising extraction quality, lowering token cost, and making retrieval and routing more dependable. Five preview capabilities move in that direction, starting with Advanced Contextualization, which underpins several of the new features.

Advanced Contextualization

Advanced Contextualization helps custom analyzers use labeled examples and document knowledge more efficiently. In internal evaluations, it improved average accuracy by up to 3.5 percent while reducing average LLM token usage by up to 22 percent. Training data remains in the customer’s Azure Storage account and is used as a knowledge source rather than copied into the analyzer, preserving customer-controlled storage. The result is higher-quality structured extraction with fewer tokens and less data-management overhead.

For more details, see analyzer improvements for more information on training data management.

The same approach is used in five new prebuilt analyzers. For these prebuilt analyzers, Advanced Contextualization reduces LLM token consumption by up to 99 percent. That makes the new prebuilt analyzers more practical for high-volume scenarios where both quality and cost matter.

The takeaway is simple: Advanced Contextualization is not just a quality feature. It is a production efficiency feature. It helps customers get better structured extraction while using fewer LLM tokens and keeping training inputs under their own storage governance.

New prebuilt analyzers for tax and other document types

This preview introduces new prebuilt tax analyzers powered by Advanced Contextualization, extending support beyond individual tax forms to enterprise and state-level tax workflows: 1065, 1120-S, 8865, 1041 Schedule K-1 and Minnesota State M1.

The new analyzers support complex, multi-page layouts and use advanced contextualization to achieve higher extraction quality, lower latency, and competitive pricing, dramatically reducing LLM token consumption, with some extraction scenarios requiring no LLM tokens at all. These prebuilt analyzers are also available in the Content Understanding Studio and Microsoft Foundry, allowing developers to visually verify values, review bounding boxes for in-document grounding, and inspect confidence scores before deploying to production. For the complete list of available prebuilt analyzers and guidance on how to use and customize them, see prebuilt analyzers documentation.

Semantic chunking in prebuilt-documentSearch

Retrieval quality often depends on how content is chunked. Fixed-size chunking can split a table from its heading, separate related paragraphs, or break a multi-page structure into fragments that are hard for retrieval systems to use.

The CU 2.0 public preview adds semantic chunking in prebuilt-documentSearch. Instead of splitting only by character count or page boundary, semantic chunking uses document structure to create more meaningful retrieval units. Semantic chunking preserves context and semantic relationships across sentences and paragraphs, and it is especially useful for RAG and agentic retrieval scenarios where the quality of what gets retrieved directly impacts the quality of reasoning outcome.

Classification: in-page splitting and confidence

Real enterprise submissions do not always align cleanly with page boundaries. A loan package, tax submission, case file, or scanned packet may contain multiple logical documents, and a single physical page can include the end of one section and the beginning of another.

The CU 2.0 public preview improves classification with in-page splitting. This allows classification to identify document segments at finer granularity than whole pages.

The preview also adds confidence for splitting and classification. Applications can use those signals to decide when to route a segment automatically, when to invoke a downstream analyzer, and when to send a result for human review.

This moves classification from a best-effort routing step toward a more operationally useful control point in document automation pipelines. For details, see classification enhancements.

Signature detection and metadata extraction in Layout

Classification decides what a document is; the Layout analyzer captures more of what is inside it. It now adds signature detection and document metadata extraction.

Signature detection helps identify signature regions and their locations in documents such as contracts, forms, invoices, and signed submissions. Metadata extraction surfaces available document properties such as author, title, creation date, content type, and language.

Combined with Layout analyzer’s existing ability to extract document structure and visual elements, including sections, headings, formatting, tables, figures, hyperlinks, annotations, and other layout elements, these new capabilities provide a more complete representation of document content and context from a single analyzer, making it easier to build intelligent document processing and agentic workflows.

To learn more, see the Layout analyzer documentation.

Try out signature detection and metadata extraction in the Content Understanding Studio.

Expanding what Content Understanding can solve

The second goal of this preview API is to reach new scenarios beyond the standard extraction. Two capabilities open that door: synchronous processing and agentic reasoning for the hardest extractions.

Read and Layout APIs

Document workflows such as grounding an AI agent during a customer interaction, validating an identity document, or triggering a workflow when a file is submitted require an immediate response. CU 2.0 preview adds synchronous operations for the Read (prebuilt-read) and Layout (prebuilt-layout) analyzers to support these low-latency scenarios. The operations return structured results directly in the response. Documents can be submitted as binary content or by URL and are processed without temporary service-side storage. To learn more, see Azure Content Understanding announces Synchronous Operations.

Agentic mode for complex field extraction

If synchronous APIs are about responding faster, agentic mode is about reasoning harder. Some document extraction tasks require more than a single pass over the content. The answer may depend on evidence spread across a long document, values may need to be compared or validated, or a field may require reasoning over intermediate results before producing a final output.

For these scenarios, the CU 2.0 preview introduces agentic mode.

Agentic mode applies an iterative extraction workflow for complex document understanding. It is designed for harder extraction scenarios where standard extraction may not be sufficient, such as long legal agreements, financial filings, insurance records, or other documents where the relevant evidence is distributed across multiple sections.

Agentic mode works with an analyzer schema and uses additional reasoning to identify relevant content, extract values, evaluate intermediate results, and refine the final output. Because it performs additional reasoning, it can increase latency and token consumption compared with standard extraction. Customers should evaluate agentic mode on representative documents and use it when the expected quality gain justifies the additional cost and processing time. Learn more about agentic mode.

How to get started

The two updates are designed to be used together: one for production today, and one for evaluating what comes next. Choose the path that matches your workload:

• Use the refreshed CU 1.0 GA API to improve existing workloads, including GPT-5 series support, grounding efficiency improvements, and the refreshed confidence model.
• Use the CU 2.0 public preview to evaluate the next generation of CU capabilities, including synchronous Read and Layout APIs, Advanced Contextualization, semantic chunking, new prebuilt analyzers, improved classification, and agentic mode.

To explore prebuilt analyzers, custom analyzers, and structured outputs, start in Content Understanding Studio or Microsoft Foundry.

• Quickstart Content Understanding Studio or Foundry | Microsoft Learn
• Quickstart: Content Understanding SDK and Rest| Microsoft Learn

The post From Sync APIs to support for the GPT-5 model series and agentic workflows: What’s new in Azure Content Understanding – August 2026 appeared first on Microsoft Foundry Blog.

Enterprise content is no longer just something people consume. As organizations increasingly rely on AI to extract and act on information from documents, images, audio, and video, Azure Content Understanding is expanding support for the GPT-5 series and improving grounding and confidence to deliver greater flexibility, efficiency, and quality.

This expanded model catalog enables organizations to choose the right level of intelligence for each workload, helping reduce costs for high-volume processing while preserving access to advanced reasoning capabilities where needed. It also provides optimized pipelines tuned for each model. Preprocessing allows the models to support larger files and higher quality than the simple LLM document pipelines. Generating grounding and confidence scores enables automated validation and higher straight-through processing rates. It also provides a clear path forward as older foundation models retire, allowing customers to transition to newer generations of models without redesigning their Content Understanding workflows.

What’s New

This release expands Content Understanding support to the GPT-5, GPT-5.1, GPT-5.2, GPT-5.4, and GPT-5.5 series including standard, mini, and nano models across document, image, video, and speech analysis.

Just as important, the release introduces an updated grounding and confidence scoring method that generates higher quality outputs and reduces overall cost. In our tested configurations, it consumed up to 28% fewer total inference tokens and the full-inference LLM cost decrease by up to 25% while improving confidence scores accuracy and grounding accuracy by up to 14% and 3% respectively (measured by AUROC and grounding exact match).

Choosing the Best Model for Your Task

Think of model selection as a mixing board, with quality on your content and end-to-end cost as the two faders to adjust. A model that excels on forms may not lead on other tasks such as video segmentation, speech classification, or image generation. As organizations increasingly leverage AI to extract information from content selecting the right model is critical for balancing accuracy, cost, latency, throughput, compliance, and regional deployment requirements.

While we cannot benchmark every combination of input types, schema definitions, and deployment topologies, below is a set of starting points and recommendations based on our testing of the most common scenarios.

We encourage customers to leverage the table above to choose a model short list, and then evaluate on your own data to make the decision. Meanwhile, you should consider other factors such as your budget, quality bar, regional availability, throughput target, and existing capacity.

Table 1: General Model Selection Guidelines

¹ Note: Speech and document extraction are similar tasks, so we currently recommend the same models for both.

Detailed model quality and cost analysis is included in the Appendix.

Grounding and Confidence improvements

In this release, we also improved grounding efficiency and refreshed the underlying confidence scoring method, so customers can get more useful evidence and ranking signals without building complex post-processing systems.

Grounding: Fewer Tokens Consumed

The updated grounding system more efficiently identifies the source for the extracted data. Across all model types, we observed 20-30% fewer input tokens and 18-28% fewer total inference tokens per document.

Those reductions lowered the full-inference LLM bill by 11-25% across the tested model-and-labeled-sample configurations and reduced P50 and P95 latency. Grounding accuracy remains similar to the previous method.

Confidence: Improved Accuracy and Generalizability

We crafted a new confidence scoring method that is applicable to a broader set of models. Measured by AUROC, which measures how reliable the confidence scores rank correct fields above incorrect fields across various threshold, the new confidence scoring system improved by about 9% for GPT-4.1 and 14% for GPT-5.2 against the previous method.

A field’s baseline confidence score combines several inputs, and score distributions can differ by field type. For straight-through processing, set acceptance thresholds field by field and recalibrate them whenever you switch models.

Appendix: Detailed Model Quality/Cost Analysis

In this Appendix, we present more details on the model quality and cost analysis conducted in-house. These results may not be representative for every production workload, but it should be helpful to the reader as a guidance for model selection.

Documents/Speech: Finding the Right Balance

Dataset

The evaluation dataset span from structured to semi-structured to unstructured documents, with a total of 71 document types. Since Content Understanding supports field extraction with and without labeled samples, we tested configurations where there are zero, one, five, and all available labeled samples, and average across them to calculate the average accuracy of a given model.

Quality is reported as a macro average across leaf fields and analyzers, excluding container fields such as arrays and objects. The evaluated documents averaged 3.35 pages. Note workloads with substantially longer files, different schemas, or different training-example strategies may see a different cost and quality frontier.

We use these document results to guide the current speech recommendation because both tasks extract structured fields from source content. We are not publishing a separate speech accuracy graph.

Results

In the figure below, GPT-5.1 and GPT-5.2 stand out as well-balanced models between AI quality and cost, and they differ by about 1% in Answer Match score. At the top of the quality range, GPT-5.5 delivers about 2% more Answer Match than GPT-5.2, but increases estimated average cost by about 101%.

Compared with GPT-5.2, GPT-5.4 Mini reduces estimated cost by about 48% while giving up about 2% in Answer Match. We recommend customers to start with GPT-5.1/5.2 or GPT-5.4 mini, then add GPT-5.5 when its quality improvement can justify the premium.

Video: the Biggest Model is not the Best Model

Dataset

The video evaluation combined two distinct workload shapes: a segmentation-focused, 60-minute video dataset, and a short whole-video dataset averaging just under one minute. We compute generation F1 score covering both scalar answer fields and timestamped custom segments, e.g., semantically relevant time windows such as when a logo appears on screen, the duration of a news segment, or an ad break.

This benchmark is most relevant to workloads that extract both video-level facts and segments from long media. Short clips, different frame density, or schemas without segmentation may produce a different ranking.

Results

As shown in the figure below, GPT-5 reached 89.0% overall Generation F1, GPT-5.1 reached 88.6%, and GPT-5.4 reached 87.5%. GPT-5 and GPT-5.1 form both the balanced and best-quality pair for this use case; GPT-5 also cost about 18% less than the GPT-4.1 baseline in the release summary.

For a lower-cost video option, we recommend customers start with GPT-5 Mini. It cost about 28% less than GPT-5 on the segmentation-focused benchmark while giving up about 7% overall Generation F1.

GPT-5 matches baseline video segmentation quality at 18% lower benchmark cost

Images: Select Different Models for Classify and Generate tasks

Dataset

The image evaluation covered 17 zero-shot datasets with five repeats. The set spanned classification and generative tasks across product, apparel, industrial-defect, scene, and object-oriented content.

Classification used F1 as metrics. Generative fields used a 1-7 rubric score, so the two quality axes answer different questions and should not be collapsed into one winner.

Results

As shown in the figure below, GPT-5.1 led classification at 69.5% Classify F1, making it an attractive choice for that use case. For generation, GPT-5.5 led at 5.0 out of 7, about 3% higher than GPT-5 Mini at about 375% higher cost.

For a lower-cost classification option, GPT-5 Mini cost about 52% less than GPT-5.1 while giving up about 12% Classify F1. We recommend customers to start with GPT-5.1 for classification-led workloads, and GPT-5 Mini for generation-led workloads. If necessary, test if GPT-5.5’s quality gain could justify the premium.

So what should teams do next?

The expanded catalog gives every channel on the mixing board a wider range. The next step is to choose two or three candidates for your workload and test which combination of quality, cost, availability, and throughput deserves the final setting.

Keep the comparison simple: use the same analyzer, schema, representative input set, and labeled examples for every run. Change only the model deployment, then compare output quality, latency, token usage, and failure rate. Before testing, check the analyzer’s supportedModels response and confirm that each candidate is available in your region.

Start in Content Understanding Studio

1. In Content Understanding Studio, open Settings, add your Foundry resource, and configure its default model deployments. Studio can deploy required models automatically when no suitable default exists.
2. Follow the Content Understanding Studio quickstart to select an analyzer and run it on your own representative content.
3. Test each candidate against the same files. Review the extracted fields and raw response, and record the quality, latency, and usage that matter to your workload.

Or compare deployments through the REST API

For a repeatable evaluation, pass a different modelDeployments mapping in each analyze request. A request-level mapping overrides the resource defaults, so you can keep the analyzer and inputs unchanged while swapping the completion deployment.

For example, you can call:

POST /contentunderstanding/analyzers/myInvoice:analyze

{
  "inputs": [
    {
      "url": "<representative-input-url>"
    }
  ],
  "modelDeployments": {
    "prebuilt-analyzer-completion": "<candidate-deployment-name>",
    "prebuilt-analyzer-embedding": "<embedding-deployment-name>"
  }
}

Run the same request once per candidate, then compare the extracted results and the response’s usage data. Start with the balanced recommendation, add the lower-cost option, and include the best-quality model only when the remaining accuracy gap matters to the workflow.

Additional Links

• For more on new capabilities in the latest Content Understanding Preview release – From Sync APIs to support for the GPT-5 model series and agentic
• To try out Content Understanding in CU Studio – Quickstart Try out Content Understanding Studio or Foundry portal – Foundry Tools | Microsoft Learn
• To use the CU APIs and SDK see – Quickstart: Azure Content Understanding in Foundry Tools – Foundry Tools | Microsoft Learn

The post Azure Content Understanding GPT-5 Series Guide: Model Selection, Grounding Improvements, and Confidence Enhancements appeared first on Microsoft Foundry Blog.

Agentic reasoning loops promise better AI answers, but could dramatically increase token consumption and costs as enterprise adoption grows.

The post Agentic Reasoning Loops Could Drive a 17x Increase in AI Token Consumption appeared first on Cloud Wars.