Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
165151 stories
·
33 followers

What's New In .NET 11?

1 Share
Read the whole story
alvinashcraft
just a second ago
reply
Pennsylvania, USA
Share this story
Delete

Claude on SQL Server FAQ: 7 Questions Teams Ask Me

1 Share

This Claude on SQL Server FAQ collects the questions teams ask me before we set up the loop. My last post, Claude on SQL Server: Read, Prove, Then Change, laid out the loop. Since then, teams have hired me to build it with them. The same seven questions come up in almost every first meeting.

Tilt-shift miniature photo of an antique brass calculating machine inside a glass museum case behind a red velvet rope, with a large magnifying glass looking in without touching it

1. Is It Safe to Let Claude Near Production?

Yes, as long as production only ever sees Claude’s reads. The guardrail is the claude_reader login, not the prompt. It holds VIEW SERVER STATE, VIEW ANY DEFINITION, VIEW DATABASE STATE and SHOWPLAN, and nothing else.

In my test on SQL Server 2025, every SELECT on a table failed with Msg 229. Run the same check with EXECUTE AS LOGIN before you hand the login over.

2. Can Claude See Our Customer Data?

Not the table rows. It can see query text, though, and query text carries values. In my test, the Query Store text of one call showed the date I typed. A literal customer number in a WHERE clause would show up the same way.

So treat the morning reports as internal documents and keep them inside the team. Mask the staging copy where your rules require it. If your work is regulated, bring your compliance person in early, when you decide what Claude can read.

3. We Don’t Have a Staging Server. Can We Still Start?

Yes. Restore last night’s backup to any spare machine with enough disk. Developer edition is free for non-production work, so licensing won’t hold you up.

What you lose is live traffic. A spare box won’t match production’s memory, load or concurrency. That’s why a gain on staging counts as evidence, and Claude checks it held on production the next morning.

Tilt-shift miniature photo of a replica brass calculating machine on a workshop bench with neat tools and a blank notebook, while the original stays in its glass case in the next room

4. How Much of Our Team’s Time Does It Take?

About 20 minutes a day, an hour a week and half a day a month. Daily, someone reads the morning report and picks one or two items. Weekly, the team replays the packs and books a change window. Monthly, they refresh staging and retire old packs.

The first week takes more, because you’re building the login, the staging copy and the first replay pack. After that, the daily part settles into those 20 minutes.

5. What Happens When Claude Is Wrong?

Some findings will be wrong. Every finding comes with a test, so a wrong idea fails on staging, not in production. It still gets a replay pack, so nobody tests the same idea twice.

Each week, note where Claude missed and add a line to its instructions. Some teams also send the same evidence to Codex for a second opinion. When the two disagree, a DBA settles it.

6. Will This Replace Our DBA?

No. Claude gathers evidence and tests ideas on staging. The DBA decides what ships, books the window and keeps the rollback ready. In a real outage, the DBA acts first, because a review step is too slow.

I do raise one warning with every team. A team that approves findings without reading them is building a new risk. Someone has to understand why each fix works.

Tilt-shift miniature photo of a curator's desk at evening with an open blank ledger, a brass key on a wall hook and a warm desk lamp

7. How Soon Will We See a Result?

Aim for one tested fix by the end of the first week. That’s the first stage: one reviewed fix, one replay pack and a morning report someone reads every day. The habit takes a few more weeks to settle.

The bigger payoff comes later, from the past cases library. Before a new investigation, Claude reads past cases that touched the same query or table. New people learn the system by reading it, instead of asking around.

Where to Start

If your team has one hour this week, create the read-only login and test it with EXECUTE AS LOGIN. Everything else builds on that. The full steps are in the first post. You can also hand that post to Claude and let it walk you through. If you’d rather have it running in weeks than months, my consulting page has the details.

I wrote about the risk in question six in Competence Debt: The Line Item No One Tracks. It’s an essay about the skills a team stops building when AI does the work. It’s one of thirty essays in my book AI: Nobody’s in There. But we’re still in here. All of them are free to read online, and the paperback is on Amazon.

The read-only login is not a limit on Claude, it is the reason teams let it in.

Published by Pinal Dave on SQLAuthority. More of my work at pinaldave.com.

First appeared on Claude on SQL Server FAQ: 7 Questions Teams Ask Me

Read the whole story
alvinashcraft
just a second ago
reply
Pennsylvania, USA
Share this story
Delete

Task queues in Durable Task Scheduler

1 Share

Task queues in Durable Task Scheduler: control the pace of your work 

As teams scale their workflows on Durable Task Scheduler (DTS), many of them hit the same problem: the scheduler is fast, but the systems they call are not. 

A batch job starts 50,000 orchestrations. Each one calls a partner API that allows only 10 requests per second. Your workers are happy to go as fast as the scheduler hands them work, so the partner API starts returning 429 Too Many Requests. Retries pile up, the batch slows down, and other workloads in the same task hub feel the pain too. 

Without a built-in way to pace one workload, teams use workarounds. Some move the slow workload to its own task hub with fewer workers. Others write throttling code inside every activity, or cap worker concurrency and hope the number matches the downstream limit. 

Task queues, now in public preview, are a simpler option. A task queue groups orchestration and activity work inside a task hub and sets how fast the scheduler hands that work to workers. 

What is a task queue? 

A task queue is a small Azure resource that lives under a task hub. It has a name and one optional setting: a throughput limit. 

Scheduler 

└── Task hub 

    ├── Task queue: orders        (20 actions per second) 

    └── Task queue: partner-api   (5 actions per second) 

You send work to a queue with a tag. When you start an orchestration (or call an activity), you set the tag DTS_TaskQueue to the queue name. The scheduler then delivers that work to workers at about the rate the queue allows. 

You keep your existing workers, client, and task hub. The only worker change is the filter setting covered in step 4 below. 

How it works 

The throughput setting is called actionsPerSecond. It controls the sustained rate of work-item handouts, which is the number of times per second the scheduler gives a work item to a worker. Here is what counts as one handout: 

  • Each orchestration turn is one handout. An orchestration usually runs in several turns: it starts, waits for an activity, then continues. 
  • Each activity execution is one handout. If an activity is delivered again after a failure, the redelivery counts too. 

Orchestrations and activities in the same queue share one rate. Activities and sub-orchestrations use their parent's queue unless you choose a different one. Entity work items don't use task queues, so if an orchestration in a queue calls an entity, the entity work runs outside the queue's budget. 

So actionsPerSecond: 20 means the scheduler hands out about 20 work items per second from this queue. It doesn't mean 20 orchestrations finish per second. Short bursts above that rate can happen, and real throughput can be lower if your workers are busy. 

If you create a queue without a throughput value, it has no queue-specific limit. Normal scheduler and worker limits still apply. 

Behind the scenes 

Clusters and partitions 

A Durable Task Scheduler runs on a cluster of backend nodes. Each task hub is split into a fixed number of partitions. A partition is a slice of the task hub: it owns a set of orchestration instances, their state, and the queues of work that are ready to run. At any moment, each partition is owned by one node in the cluster. If a node fails or the cluster rebalances, another node takes over the partition and continues from its saved state. 

When you start an orchestration, the scheduler hashes the instance ID to choose its partition, so the same ID always lands in the same partition. With many instances and random IDs, work spreads evenly across partitions. 

Inside each partition there is a base queue for normal work, plus one ready queue for each task queue you define. Workers don't connect to partitions directly. A worker opens one streaming connection to the scheduler, and the scheduler pulls work for that worker from every partition. 

 How work flows through a Durable Task Scheduler 

 

 

The routing decision is saved with the instance 

When an orchestration starts with DTS_TaskQueue, the scheduler saves the queue name as part of the instance's state. Later turns of that orchestration, the activities it schedules, and the child orchestrations it starts all go to the same queue unless you override it. Because the choice is stored with the instance, it survives node failovers and retries. 

 

 

 

The queue choice is saved with the instance 

This design has three effects: 

  • When an external event, an activity result, or a child result wakes up an orchestration, the work goes to that orchestration's queue, not the sender's. 
  • Continue-as-new can switch queues. If the new execution has a DTS_TaskQueue tag, that becomes the new choice. If it doesn't, the old choice stays. 
  • The scheduler saves the queue name even if the queue doesn't exist, and checks whether it exists each time it routes work. If the queue is missing, the work goes to normal routing and the scheduler logs a warning. The work itself doesn't fail. 

A token bucket in every partition 

The rate limit is a token bucket. Every handout from a limited queue takes one token, and tokens refill with time, not when work completes. When the bucket is empty, the work waits in its queue while other queues keep moving. 

The rate you configure applies to the whole task hub. The scheduler divides it evenly across partitions, so each partition has its own small bucket for each queue and decides on its own, with no central coordinator. 

 

 

 

One rate, split across partitions 

For example, with actionsPerSecond: 20 and a task hub with 4 partitions: 

  

Value 

Refill rate per partition 

20 ÷ 4 = 5 tokens per second 

Bucket size per partition 

1 second of its share = 5 tokens 

Total across the task hub 

4 × 5 = 20 handouts per second 

 

Buckets start full. When a partition starts or moves to a new node, you can see a short burst before the steady rate takes over. 

Low rates still work. A partition's share can be less than one token per second; the scheduler keeps the fraction, and each bucket can hold at least one token. That partition waits longer between handouts. 

The split is static, so partitions don't borrow tokens from each other. If most of your work sits in one partition, it can only use that partition's share while the other buckets stay full. This matters when you have only a few long-running instances. With many instances, the hash spreads them out and you get close to the full rate. 

Taking turns between queues 

When a worker asks a partition for work, the partition has several ready queues: the base queue and each task queue. The scheduler reads from all of them at once and takes the first item that is ready. If more than one item comes back, it keeps one, puts the others back, and returns their tokens to the bucket. 

The starting queue rotates on each read (round-robin), so every queue gets a turn to be first. A busy queue can't block a quiet one, and no queue has priority over another. 

Worker concurrency limits still apply. A token lets the scheduler hand out a work item, but the worker still needs a free slot to accept it. 

Changing queues while work is running 

Queue definitions are Azure resources managed by the DTS control plane. When you create, update, or delete a queue, a durable control-plane workflow saves the new definition. Every partition then reloads its list of queues, and worker connections start reading the new set. 

Updating a rate gives each partition a new bucket with the new refill speed, and like any new bucket it starts full. Work is not moved or cancelled. Deleting a queue doesn't wait for it to drain: its pending work moves to normal routing, and work that is already running can finish. 

Example: an order workload with a rate-limited partner API 

This example processes orders and calls a partner API that allows 10 requests per second. 

Step 1: Create the queues 

Task queues use management API version 2026-07-01-preview. You can create them with the Azure CLI, the ARM REST API, or Bicep. Here is Bicep: 

 

param schedulerName string param taskHubName string resource scheduler 'Microsoft.DurableTask/schedulers@2026-02-01' existing = { name: schedulerName } resource taskHub 'Microsoft.DurableTask/schedulers/taskHubs@2026-02-01' existing = { parent: scheduler name: taskHubName } resource ordersQueue 'Microsoft.DurableTask/schedulers/taskHubs/taskQueues@2026-07-01-preview' = { parent: taskHub name: 'orders' properties: { throughput: { actionsPerSecond: 20 } } } resource partnerQueue 'Microsoft.DurableTask/schedulers/taskHubs/taskQueues@2026-07-01-preview' = { parent: taskHub name: 'partner-api' properties: { throughput: { actionsPerSecond: 5 } } }

Or with the Azure CLI: 

taskHubResourceId="/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.DurableTask/schedulers/<scheduler>/taskHubs/<task-hub>" queueUrl="https://management.azure.com$taskHubResourceId/taskQueues/orders?api-version=2026-07-01-preview" az rest --method put \ --url "$queueUrl" \ --body '{"properties":{"throughput":{"actionsPerSecond":20}}}'

Wait until properties.provisioningState is Succeeded before you send work to the queue. 

 

Step 2: Route an orchestration to a queue 

Add the DTS_TaskQueue tag when you schedule the orchestration: 

var options = new StartOrchestrationOptions { Tags = new Dictionary<string, string> { { "DTS_TaskQueue", "orders" }, }, }; string instanceId = await client.ScheduleNewOrchestrationInstanceAsync( "ProcessOrderOrchestration", input: order, options: options);

The same idea works in Python: 

instance_id = client.schedule_new_orchestration( "process_order_orchestration", input=order, tags={"DTS_TaskQueue": "orders"}, )

Every activity and sub-orchestration this orchestration starts will also use the orders queue. You don't need to repeat the tag on each call. 

Step 3: Slow down only the part that needs it 

Often only one step of a workflow needs a lower rate. Here, the orchestration can move at 20 handouts per second, but the partner API call should stay at 5 per second, well below the partner's limit of 10. You can move just that activity to another queue: 

var partnerOptions = new TaskOptions(tags: new Dictionary<string, string> { { "DTS_TaskQueue", "partner-api" }, }); await context.CallActivityAsync( "CallPartnerApi", input: order, options: partnerOptions);

The parent orchestration stays in orders. Only CallPartnerApi is paced by partner-api. 

Step 4: Make sure your workers can pick up the work 

Only workers without work item filters receive work from a task queue. The .NET SDK creates filters automatically from your task registry, so turn them off for these workers: 

builder.Services.AddDurableTaskWorker() .AddTasks(registry => registry.AddAllGeneratedTasks()) .UseDurableTaskScheduler(connectionString) .UseWorkItemFilters(null);

In the Java, Python, and Go SDKs, don't turn on work item filtering for these workers. 

Step 5: Tune at runtime 

You can change the rate at any time without redeploying code or changing tags: 

az rest --method patch --url "$queueUrl" \ --body '{"properties":{"throughput":{"actionsPerSecond":10}}}'

To remove the limit but keep the queue, set throughput to null. 

Use cases 

Rate-limited downstream services. A payment provider, a CRM, or an LLM endpoint often has a strict quota. Put the activities that call it in their own queue and set a rate below the quota, so fewer calls come back with 429. 

Large batch jobs. Nightly imports and data backfills often start thousands of orchestrations at once. A task queue lets the batch move at a steady pace instead of hitting your database or storage in one wave. 

Mixed workloads in one task hub. If a heavy workload shares a task hub with lighter, interactive work, a rate limit on the heavy workload's queue leaves room for the rest, without splitting into separate task hubs. 

AI agent workflows. Agent loops can call models and tools many times per request. Routing the model-calling activities through a queue helps you stay inside token or request quotas while the rest of the workflow keeps moving. 

Limits to know 

Task queues pace work. Keep these limits in mind when you design with them: 

  • Queues have no priority. Each queue is paced on its own, and none runs before another. 
  • Queues don't reserve compute or give physical isolation. They share the same scheduler and workers. 
  • The handout rate is not an exact cap on downstream calls, because of bursts and retries. For a hard limit, also add rate limiting in the activity that calls the service. 
  • If the tag names a queue that doesn't exist or was deleted, the work still runs, but without the rate limit. Don't delete a queue to pause a workload. 
  • Work in a defined queue waits for an unfiltered worker. It doesn't fall back to a filtered one. 
  • A queue doesn't limit how fast you create instances. If you start 50,000 orchestrations, they are all created right away, and their work then waits in the queue. 
  • In this preview, extended sessions are not paced. If your workers use them, turns that run inside an open session don't take a token. 

SDK support 

You select a queue with orchestration and activity tags. Support today: 

SDK 

Start orchestration 

Call activity 

Call sub-orchestration 

Durable Task .NET 

Yes 

Yes 

Yes 

Durable Task JavaScript/TypeScript 

Yes 

Yes 

Yes 

Durable Task Python 

Yes 

Yes 

No 

Durable Task Java (v1.6.0+) 

Yes 

No 

No 

Durable Task Go (beta) 

Yes 

Yes 

Yes 

Durable Functions .NET isolated 

Yes 

Yes 

Yes 

Durable Functions Java 

Yes 

No 

No 

 

When an SDK can't set the tag on a child call, the child inherits the parent's queue. 

Best practices 

Start with one queue. Put the workload you worry about most into a single queue with a safe rate. Add more queues only when you see a clear need. 

Think in handouts, not orchestrations. An orchestration that calls five activities one after another uses about 11 handouts: six orchestration turns (one to start, plus one after each activity) and five activity executions. Do this math before you pick a number. 

Set the rate below the downstream limit. Leave room for retries and short bursts. 

Spread the work. The rate is split across partitions, so it works best with many instances. If you run only a few long-lived orchestrations, expect lower throughput than the configured rate. Random or unique instance IDs help the hash spread work evenly. 

Create the queue first, then send work. Wait for provisioningState to be Succeeded. Otherwise the work runs without a limit. 

Check the dashboard. The DTS dashboard shows orchestration status and tags, so you can confirm that work lands in the queue you expect. 

Getting started 

Task queues are available now in public preview. To try them: 

  1. Create a queue under your task hub with API version 2026-07-01-preview. 
  1. Make sure your workers connect without work item filters. 
  1. Add the DTS_TaskQueue tag when you schedule work. 

Read the full guide: Use task queues in Durable Task Scheduler. 

To tell us how you use task queues or what you want next, open an issue in your SDK's GitHub repository. 

 

Read the whole story
alvinashcraft
2 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Announcing Public Preview of the next generation of Azure CLI and Azure PowerShell

1 Share

Today, we are announcing the public preview of the next generation of Azure CLI and Azure PowerShell: a shared command engine with new experiences for discovering commands, managing sign-in contexts and troubleshooting failures.

Whether you are managing Azure resources from the terminal or automating recurring tasks in scripts and pipelines, the goal is practical: spend less effort assembling the command and more time accomplishing the task. Install the preview alongside your existing tools and start with one workflow.

Fundamental shift 

The Azure CLI and Azure PowerShell you have been using were built in a world where the cloud had a lot less services than today, security requirements and speed of development were different.

To meet our customer’s expectations in today’s world, we are shifting how we are building the command line tools for Azure. Our approach is based on three pillars:

  • Parity betwen native command and PowerShell cmdlets
  • Meet modern security and safety requirements
  • Build for humans and agents

The new foundation is designed for a smaller disk footprint, more responsive commands and faster delivery of fixes and features. Azure CLI offers a smaller base (lean package) that can grow through service-specific extensions, while Azure PowerShell supports individual service modules rather than requiring the entire bundle.

Keep your work environments within reach

Save a sign-in context instead of rebuilding it for every task. Named profiles capture the tenant, subscription and authentication context for an environment. Switch between saved profiles, or use a different profile for one CLI command without changing the context for subsequent commands. For example: Use your Test profile to inspect a resource in a test subscription, while keeping Development as the default for subsequent commands. Profiles make the context explicit; Azure permissions still govern access.

Automate without storing secrets. Scripts and pipelines can sign in with managed identities or federated credentials, so there is no application secret to store, protect or rotate.

Watch: Introduction to Profiles

 

Find the command without the guesswork

Discover required inputs and examples where you work. The new CLI help experience organizes command information so you can move from a command group to the operation you need. Tab completion helps you discover command names and parameters as you type.

For example, when creating a resource group, you can inspect the required name and location and see an example before making a change. External AI agents can consult the same help, constructing commands from current details rather than relying only on previously learned syntax.

Turn a command failure into a next step

Get troubleshooting guidance without leaving the terminal. Optional AI-assisted troubleshooting in the CLI analyzes a failed command and its error, then offers an explanation and suggested actions.

For example, if VM creation fails because the requested size is unavailable in the chosen region, AI-assisted troubleshooting can explain the restriction and guide you through checking alternatives. You review the options before changing the size or location.

In this preview, AI troubleshooting is available in Azure CLI and is off by default. You can request diagnosis manually, enable automatic diagnosis on failures, or turn it off.

Watch: AI-Assisted Error Recovery

Help shape what comes next

This public preview is an opportunity to validate the fundamentals with your feedback. Preview coverage is limited to a subset of commands and scenarios, so it is not yet a replacement for your existing tools, and behavior may change during the preview. Check supported scenarios and known issues https://learn.microsoft.com/cli/azure/whats-new?view=azure-cli-preview before you start.

Moving forward, we will focus on expanding coverage across Azure services and everyday workflows, guided by your feedback. We will also build on these foundations with deeper AI-agent readiness, making supported commands easier for agents to discover, interpret and use reliably.

Choose the experience you use most:

Try the Quick start guides. Tell us what was clearer, where you encountered friction and which missing capability would make the biggest difference.

Read the whole story
alvinashcraft
2 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Hosting your LangGraph application on Microsoft Foundry Agents

1 Share

This post was co-authored my Maira Ladeira Tanke and Muzz Imam

 

Consider a sales team that wants an assistant to handle customer returns. A customer asks, “Can I return my order?” The assistant needs to understand the request, look up the order, and use the available tools to help the customer through the return process.

The engineering team builds it in LangGraph, and for the rest of this post, we’ll take their seat as we bring it to production on Microsoft Foundry Hosted Agents without rewriting the graph. The graph manages the conversation, calls the model, and connects to the order and return tools. For this example, the architecture is straightforward:

 

The prototype works well. We test different return scenarios, demonstrate the assistant to leadership, and get approval to make it available to customers. Now we need to run it in production.

We start looking at hosting options: AKS, Azure Container Apps, Azure Functions. But agentic applications have their own requirements, and choosing where the code runs needs to consider them. Our sales assistant needs to support multi-turn conversations, keep concurrent customer conversations isolated, and preserve context when a customer leaves halfway through a return and comes back the next day. These requirements need to be part of the hosting decision from the start.

Where do we store conversation history and the state of an unfinished return? How do we connect Microsoft Entra ID and control access to the order system? How do we screen harmful content and check that the assistant follows the return policy as we change prompts or models? When a customer gets the wrong answer, can we inspect the model and tool calls that led to it? We need to plan capacity and scaling, handle concurrent requests, and deploy new versions without interrupting conversations already in progress. If another team wants to reuse the return tools, we need a way to share and manage access to them too.

None of this means we chose the wrong framework. The LangGraph application is doing what we built it to do. But the production plan now includes a conversation store, identity integration, tracing, deployment policies, and infrastructure decisions. We need to decide how much of that platform our team wants to assemble and maintain.

This is where Microsoft Foundry Hosted Agents enters the discussion. We can keep our LangGraph application and run it in a managed agent runtime. In this post, we will follow the sales assistant into production and look at what Foundry takes on, what we still need to configure, and what remains our responsibility.

Deploying your agent to Foundry Hosted Agents

The good news is that we do not need to rewrite the sales assistant to run it on Foundry Hosted Agents. We keep the graph, prompts, and order and return tools we already built. LangGraph still controls the conversation and the application logic. The hosting integration exposes our existing compiled graph through an endpoint that Foundry can serve; it does not require us to move to a different agent framework. The Learn hosted-agent quickstart covers the deployment workflow.

The langchain-azure-ai package is the entry point for building LangChain and LangGraph applications with Microsoft Foundry capabilities, including models, tools, and tracing. For hosting, we use its hosting integration to connect the existing graph to the managed runtime. We still need to configure dependencies, credentials, and deployment settings, but the assistant itself remains a LangGraph application.

We can expose the agent through either the Responses or Invocations protocol. Responses provides an OpenAI-compatible chat endpoint, including streaming and conversation threading. For our sales assistant, this is a natural starting point. With the hosting dependencies installed, we pass the existing compiled agent to ResponsesHostServer:

 

from langchain_azure_ai.agents.hosting import ResponsesHostServer # your agent code remains unchanged # ..... ResponsesHostServer(agent).run()

ResponsesHostServer is currently in preview.

If the application needs a custom JSON request and response shape instead, we use the Invocations protocol. The setup follows the same pattern, using InvocationsHostServer in place of ResponsesHostServer:

from langchain_azure_ai.agents.hosting import InvocationsHostServer # your agent code remains unchanged # ... InvocationsHostServer(agent).run()

These are alternative entry points for the same agent logic. We choose the protocol that fits the application calling it; neither option requires rewriting the graph.

For an existing project with a graph referenced in langgraph.json, we can also use the langchain_azure_ai.agents.hosting.run module without adding a server entry-point file. In the Azure Developer CLI configuration, we set codeConfiguration.entryPoint in azure.yaml to run that module with the chosen protocol. For Responses, the relevant configuration looks like this:

services: my-agent: host: azure.ai.agent kind: hosted env: PYTHONNOUSERSITE: "1" codeConfiguration: runtime: python_3_13 entryPoint: '-m langchain_azure_ai.agents.hosting.run --protocol responses'

What changes once the agent is deployed

Our sales assistant is now serving customers through a Foundry Hosted Agent endpoint. The LangGraph application still handles the conversation and calls the order and return tools. Around that code, we now have sessions, conversation history, and a place to inspect execution. The architecture looks like this:

You pay for session CPU and memory only while compute is active, with no compute charge once the session scales to zero after its idle timeout; model, tool, and other Azure service charges are separate.

Continuing the conversation across turns

A return rarely fits into one message. The assistant might ask for an order number, check the order, and ask which item the customer wants to return. With the Responses protocol, Foundry stores the conversation history. We pass the same conversation ID on subsequent turns so the assistant can use the earlier exchange rather than ask the customer to repeat it. Foundry also associates that conversation with a session, keeping the chat connected to the same working environment.

Foundry also provides a portal playground where we can quickly test conversations with the deployed agent. We can open the sales assistant, ask for help with a return, and follow up with the order details, all in the same chat. This gives the team a way to try the conversation before building a customer-facing interface. For our sales assistant, we can check whether it asks for the information it needs and uses the context we already provided.

A conversation and a session serve different purposes: the conversation holds messages, tool calls, and responses; the session provides the sandbox and its persisted filesystem. Reusing a session ID alone does not restore chat history. With Invocations, our application manages that history itself. Any additional LangGraph workflow state that must survive a process restart also needs durable persistence; conversation history is not a checkpoint of the whole graph.

Keeping each customer’s work separate

User A and User B can reach the same agent endpoint without sharing a conversation or workspace. For callers authenticated directly with Microsoft Entra ID, Foundry scopes sessions, conversations, and stored data to the calling identity. Each session runs in its own VM-isolated sandbox with a private filesystem. User B’s request for an update belongs to User B’s return, not the conversation User A started.

For a customer-facing application that calls Foundry through a shared backend identity, we configure end-user identity delegation and maintain separate session and conversation mappings for each customer. Foundry Toolbox lets us define a reusable collection of tools and expose it through a single MCP-compatible endpoint, with authentication and access managed centrally. For our order and return tools, this provides a shared endpoint with centrally managed credentials, versioning, and policies, while carrying user context through to supported tools. Another team can use the same tools in its own assistant instead of rebuilding the connections and authentication in each application. For tools that support on-behalf-of-user (OBO) access, Foundry handles the downstream token exchange outside our agent code. The order service receives the signed-in user’s delegated context and applies its permissions when looking up an order or processing a return. Session isolation keeps the workspaces separate; the order service continues to enforce access to the business data and actions. We configure the supported authentication method, delegated scopes, and consent. Custom MCP tools use the agent-identity OBO configuration, currently in preview. For direct API calls outside this managed tool path, our application remains responsible for consent, token acquisition, and token protection.

For the sales assistant’s order-system connection, we can configure private networking and network egress rules (preview) to restrict outbound connections to approved destinations; these controls require explicit configuration.

We also configure content safety guardrails to screen incoming prompts and outgoing responses. We create a Responsible AI policy in Foundry, attach its resource ID to the hosted-agent definition, and test both allowed and blocked requests before release. With Responses, the platform knows where to find the text to screen; with Invocations, we also configure the request and response fields to moderate. A deployed endpoint alone is not proof that filtering is active. These filters complement, rather than replace, our checks on return eligibility and authorization.

Seeing what happened inside a request

Suppose a customer says the assistant gave them the wrong return instructions. Reading the final answer is not enough to understand what went wrong. We need to inspect the request: the input the agent received, the model and tool calls it made, and the output at each step. With the OpenTelemetry integration in langchain-azure-ai, we can capture that execution in Azure Application Insights and inspect it in the Foundry portal. For the sales assistant, we can check whether it called the order lookup tool, what information the tool returned, and how the agent used that information in its response.

When we inspect a recorded request in Foundry, User view shows the exchange as a conversation, including the customer’s messages and the assistant’s replies. We can then inspect the recorded inputs, outputs, and tool execution behind that exchange. Graph view shows the relationships between the captured operations in this run. It is a view of the execution trace, not a replacement for the LangGraph application’s graph definition.

Trace Replay  lets us step through the recorded interaction in sequence, pause on a particular operation, or jump to the part we want to investigate. We can move from the answer the customer saw to the tool result behind it and review where the behavior diverged from what we expected. Replay is a way to inspect the captured run; it does not rerun the return or execute the tools again.

 

 

For our team, this turns a customer-reported issue into a support incident we can investigate. We can distinguish an incorrect tool result from an incorrect interpretation of that result, or see whether a slow response spent its time in the model or a downstream call. The observability connection assumes tracing is enabled and the relevant operations are instrumented. We also control whether message content and tool arguments are captured, and restrict access to traces that contain customer or order information.

Monitoring the agent beyond a single conversation

A trace helps us investigate one customer’s return. Once the assistant is handling many conversations, we also need to know how it is performing across those requests. Are customers waiting longer for an answer? Are more runs failing? Is token usage growing along with traffic, or has the agent started using more tokens to handle similar requests? The Foundry monitoring dashboard brings token usage, latency, and run success rate into the agent’s Monitor tab, with summaries and charts for the time range we select. This gives us an operational view across conversations, rather than only the details of one run.

For the sales assistant, we read those signals together. If token usage rises, we can investigate whether the change reflects more conversations, longer prompts and responses, or extra model calls. If response times increase or the run success rate drops, we can inspect traces from that period to understand which operations need attention. The dashboard helps us spot the pattern; the trace helps us investigate the individual execution. Token counts are useful for understanding model consumption, but they are not the full cost of running the application.

 

 

Operational health is only part of the picture. A run can complete successfully and still give the customer an incorrect answer. When configured, the Monitor experience also brings in evaluation scores and red-team results, alongside preview options for recurring evaluations and alerts. For our team, that means we can review reliability and usage alongside checks on answer quality and safety, rather than treating a successful request as proof that the return was handled correctly. Agent metrics in this dashboard are currently in preview and use the Application Insights resource connected to the project, with the corresponding access permissions and retention settings.

Production is the beginning of the improvement cycle

Getting the sales assistant into production is only the first step. To check whether it is doing its job, we define representative return scenarios and expected outcomes: following the return policy, asking for missing order details, selecting the right tools, and refusing unauthorized actions. We use Foundry evaluations to score the agent’s responses and conversations with built-in quality and safety evaluators, plus custom checks for our business rules. In the portal, we create an evaluation, select the agent and test data, and choose the evaluators that match those criteria.

Before changing a prompt, tool, or underlying model, we run the current and candidate versions against the same test cases and scoring criteria. We compare task outcomes and safety results alongside latency and token usage, then inspect individual failures rather than relying only on an average score. Production traces and insights help us identify cases to add to the test set. The agent optimizer overview explains the optional next step of generating and evaluating candidate configurations. Our team still defines success and decides which changes are ready for customers.

Let Foundry do the heavy lifting

We started with a sales assistant that handled returns well in a prototype. Making it useful to customers also meant supporting ongoing conversations, keeping their work separate, controlling access to orders, and understanding how the agent behaved. Those responsibilities remain important long after the first deployment.

Hosting on Foundry lets us keep the LangGraph application and use managed capabilities around it, rather than assemble every part of the platform ourselves. Our team still owns the return experience, the business rules, and the decision to release a change. Traces, monitoring, evaluations, production insights, and optimization give us evidence to work with as we improve that experience.

If you already have a working LangGraph agent, start with the Learn hosted-agent quickstart and explore the LangGraph examples for multi-turn chat, Toolbox integration, and observability. Test a representative return conversation in the portal, inspect its trace, and turn the expected outcome into an evaluation case. Foundry handles more of the hosting platform so our team can focus on ongoing business value: helping customers complete eligible returns correctly and safely, and checking that each release preserves or improves that experience.

Read the whole story
alvinashcraft
2 hours ago
reply
Pennsylvania, USA
Share this story
Delete

How deep do you want to go? Microsoft Copilot, Cowork, and Viva Engage in action

1 Share

There’s so much beneath the surface in Engage, including insights into:

  • How employees are feeling
  • The questions they keep asking
  • Feedback that can sharpen your next message
  • Knowledge worth sharing more widely

New Copilot in Engage and Copilot Cowork experiences help communicators, community managers, and leaders bring more of it into view, connect these insights with the work they’re already doing, then make better decisions from what they learn.

Join our webinar: Go Deeper with Engage with Copilot and Cowork
December 3, 2026, 9:00 AM PT

See Copilot in Engage, Cowork, and Community Agent in action through live demos of real scenarios. You’ll also learn how to get set up with the Engage plugin and ready-made skills so you can try it yourself right away.

What’s new

Enhanced Copilot in Engage

Coming soon, the enhanced Copilot in Engage can analyze conversations, connect Engage insights with your wider Microsoft 365 work, and creating outputs such as community health reports, posting plans, and presentations.

Engage plugin for Copilot Cowork

Rolling out now, the Engage plugin brings community knowledge, conversations, and actions into Copilot Cowork. Cowork can combine Engage content with your meetings, chats, emails, and files to complete multistep tasks on your behalf, like posting to Engage and sending emails.

Use Viva Engage in Microsoft 365 Copilot Cowork

Use plugins with Copilot Cowork

Ready-made skills for common jobs

To get you started, we’ve built skills that help community managers and corporate communicators complete repeatable work faster. Choose a workflow from the Community Pack or Comms Pack, such as reviewing how a message landed, preparing a leader briefing, or drafting a community roundup, and prompt Cowork to begin.

Get the skills and learn how to add them to Cowork

New capabilities for Community Agent

Community Agent is an AI agent available in public preview for Viva Engage communities that helps admins and experts keep up with colleagues' questions. Recent updates to Community Agent bring more knowledge sources, clearer answers, and easier draft review and activity tracking. 

Read the full roundup of Community Agent improvements

Note: the Microsoft Copilot, Cowork, and Community Agent experiences described in his blog require a Microsoft 365 Copilot license.

Go deeper: three everyday scenarios

The possibilities for going deeper with Copilot, Cowork, and Engage are endless. The following examples show how you can get deeper levels of help with your work, from exploring information to bringing together context and carrying out a task.

How did your leader’s message land?

Deep: Review an announcement’s post analytics, including sentiment and themes from comments. Gather the questions and reactions from the comments that should shape your follow-up.

Deeper: Ask Copilot in Engage to identify themes and unresolved questions and turn the findings into a presentation.

Deepest: Have Cowork prepare a repeatable leader-ready briefing that brings together post metrics, comment themes, and recommended next steps. Review it before by sending to your leader by email.

Try it in Cowork:

  1. Get the /how-did-it-land skill for Cowork.
  2. Prompt: /how-did-it-land, how did the post by [leader] about [topic] land?
  3. When you’re happy with the report, ask for an overview email you can send to the leader.
Video created with Copilot Cowork

Is your community getting the answers it needs?

Deep: Review community analytics, including sentiment and themes. browse discussions and find questions that need an answer.

Deeper: Ask Copilot in Engage to summarize recurring questions and suggest useful follow-up content. Could an FAQ or pinned “Start here” post help?

Deepest: Let Community Agent help handle questions using community conversations and configured knowledge sources. It drafts answers for review or posts automatically according to community settings.

Us the /community-report and /questions-needing-help skill to ask Cowork to give you a readout on how your community is doing and where to focus attention. Ask for an interactive dashboard to share.

Try it with the Community Agent and Cowork:

  1. Add the Community Agent to the community and provide it with relevant knowledge sources from SharePoint and public websites.
  2. Get the /community-report and /questions-needing-help skills for Cowork.
  3. Prompt: /community-report give me a readout on [community name]. /questions-needing-help Include a view of questions that haven’t been answered or need additional help. Make it an interactive dashboard I can share.
Video created with Copilot Cowork

How do you turn a busy week into a great update?

Deep: Gather meeting notes, chats, and files, then write an Engage update.

Deeper: Ask Copilot in Engage to turn that context into a draft for your audience, informed by your work and writing style.

Deepest: Ask Cowork to gather the context, prepare the post, and publish through the Engage plugin after you approve it.

Try it in Cowork:

  1. Prompt: Use my meetings, chats, and files about [project] from the last week to draft an update for the [community name]. Explain the latest status, celebrate successes, highlight what we are learning, outline next steps, and invite questions. Thank contributors. Show me the draft and publish to Engage only after I approve. 
  2. Provide feedback on the message, then approve. Cowork will post to Engage on your behalf.
Video created with Copilot Cowork

Engage skills for Cowork

We're introducing 17 skills for community managers and corporate communicators to use with the Engage plugin in Microsoft Copilot Cowork.

Start with the result you need: use /how-did-it-land to review available post metrics, recurring questions, and feedback, with a suggested next step. Try /leader-briefing to turn selected conversations into a concise private brief, or /community-roundup to draft a summary of useful conversations on a chosen topic.

How skills work

The Engage plugin lets Copilot Cowork find and read conversations in Engage. Once you add the plugin, you can bring those conversations into the work you do in Cowork. A skill is a set of instructions for a specific job. You choose the skill and provide the community and/and topic; Cowork follows the instructions and uses the plugin to get the information it needs.

 

Download the skills

 

  1. Add the Engage plugin for Cowork.
  2. Download the pack of skills, then extract the files
  3. Open Copilot Cowork. Select Customize. Select the Add menu, then Upload and choose a skill to add.
  4. Try the skill. Start a new Cowork conversation. Type / to choose an installed skill, or simply prompt in plain language. For example, "how did [leader name] message about [topic] land?"

Do you have feedback on these skills or an idea for a skill you'd like us to create? Share your feedback in a short survey.

For more setup details, see Customize Copilot Cowork and Use plugins with Copilot Cowork.

Choose your skills

Comms Pack

Use these skills to prepare communications, understand the response, and organize follow-up work.

Skill

What it helps you do

/how-did-it-land

See how an Engage post was received: its first-week numbers against your recent posts, the main themes and open questions, and one next step.

/network-themes

See the main topics people have discussed across Engage in the last 28 days, with example conversations.

/metrics-tracker

Put first-week reach, reactions and comments for the Engage posts you choose into a new or existing Excel tracker.

/ama-question-tracker

Build an Excel tracker of AMA questions grouped by theme, or safely update one this skill created.

/leader-briefing

Turn selected Engage conversations into a short private brief for a leader, with a suggested next move for each.

/leader-reply-radar

Find up to 5 conversations where a reply or thank-you would help, with draft wording to paste.

/pre-flight-check

Check a draft against its sources and get a go / no-go, each error with evidence, and the smallest fixes.

/event-close-out

Sort an event's questions into answered, partly answered and open, and draft follow-up text for the event owners.

/question-forecaster

Prepare likely questions and a draft FAQ from approved facts, flagging answers that still need a decision.

/leader-moment-composer

Turn a brief into draft of a leader post with two alternative openings and notes on where each claim comes from.

/recommended-leaders

Review Engage's recommended-leader suggestions and visible setup, and prepare an admin checklist.

/what-to-feature

Get 2-5 recent conversations worth featuring or pinning, each with a reason. You decide and take the action.

Community Pack

These workflows focus on understanding a community, helping members find useful information, and handling membership tasks.

Skill

What it helps you do

/community-review

Explains what your community contributes, how people participate, who helps keep it active, and where you could focus next.

/community-roundup

Turns useful updates, answers, resources and event takeaways into a catch-up post for members who missed the conversations.

/contributor-recognition

Helps you decide whom to recognize and why, based on helpful answers, useful resources and constructive contributions.

/questions-needing-help

Finds question posts that still need a specific answer or follow-up after reviewing the replies. Separately identifies existing answers worth reviewing for verification.

/community-membership-summary

Shows who is represented in your community through breakdowns by job title, community role and recorded office location, where available.

 

Read the whole story
alvinashcraft
2 hours ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories