Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
161233 stories
·
33 followers

Today I will… improve test coverage

1 Share

Today I want to look at how we can improve on the test coverage that we have in the Interview Coach application. This application has some unit tests in it already, but I’m sure there is room for improvement.

Defining our baseline

Before we start writing tests, it’s a good idea to have a baseline of what our coverage is so that we can find out where the gaps are, rather than just writing tests “for the sake of writing tests”.

We can do this using the Analyze Code Coverage for All Tests, from either the Command Palette (CTRL + SHIFT + P) or the Tests menu.

Feature Search for test coverage

Running the analyzer, we can see where our coverage is sitting:

Test Coverage results

It looks like our coverage isn’t great, there’s a lot of gaps in the InterviewCoach.Agent project, and then other projects, such as InterviewCoach.Mcp.InterviewData, which doesn’t even have any tests. Let’s start with some tests for the InterviewSessionTool class within that project.

Writing tests with Copilot

The class we’re going to write tests for is a simple class, it takes an interface of our repository, so we can mock the database calls, and the methods receive a payload to work with the repository.

To speed things along, we’re going to use the Test Agent from GitHub Copilot and provide it with the file we want the tests written for.

Test Agent usage

The agent will analyse the class, then the solution to find the right approach to writing tests. Since we don’t have a test class for the project yet, it’ll generate one, add the references, and then write our test cases.

After a few minutes the agent is done.

Test Agent output in Copilot Chat

It gives a nice summary of what’s been completed, we have 8 new tests added, along with a new test project. The tests were run, and it’s suggesting we run the coverage report again.

Updated coverage report

This is a considerable improvement on where we were at beforehand.

Full solution test analysis

The Test agent has done a great job at building out the tests for our class, but there’s still a lot of the codebase that doesn’t have tests, and we can use the Test agent to help with that because we can tell it to run across the whole solution and analyze our gaps:

@test #solution

This time, we’re using the #solution context for Copilot, and now the Test agent will analyze all projects and code paths. After a few minutes, Copilot has come back to me with tests covering the repository, the DbContext, our extension methods, as well as more tests in the InterviewCoach.Agent project. It also identified that there was no value in writing tests for files such as AgentMode.cs since it just contains an enum, thus there’s nothing to test.

With this, our coverage is up to 81%, up from 37% to start with, and includes three projects we originally didn’t have tests for at all.

Wrap-up

While we know the value of writing tests, knowing where we should invest that effort is something that is often overlooked. Using the Code Coverage analysis in Visual Studio gives us that overview, so we can identify the gaps in our testing, then combining that with the Test agent, we can build out targeted tests for a single class or perform a solution-wide analysis and uplift.

The post Today I will… improve test coverage appeared first on Visual Studio Blog.

Read the whole story
alvinashcraft
13 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Build an interview coach app with the GitHub Copilot SDK

1 Share

An interview coach has to do more than ask questions. It needs to read a resume, follow up on an incomplete answer, and save enough context to give useful feedback at the end. Some of that work is conversation. Some of it requires calling an application service.

The GitHub Copilot SDK lets you use the runtime behind Copilot CLI for that work inside your own application. You provide instructions and callable tools. Copilot handles the model interaction and resulting tool calls, while your application owns the interface and business workflow.

For a developer building a personal assistant or an internal workflow, this means reusing an agent runtime with capabilities specific to the application. Existing Copilot access is a practical benefit, too: the sample’s local Copilot configuration doesn’t need a separate Azure model deployment.

Interview Coach demonstrates this in .NET. The candidate sees a Blazor chat interface. Copilot receives interview instructions and tools for handling documents and session records. We’re using Copilot to run part of the app, not to edit its code.

Interview Coach uses the GitHub Copilot SDK for model interactions and tool calls, with a Blazor interface, Agent Framework handoffs, MarkItDown MCP, and InterviewData MCP backed by Cosmos DB.

What the candidate experiences

You provide a resume and job description, answer behavioral and technical questions, and receive feedback on your responses. When you finish, the coach reviews the interview record and produces a summary.

For example, imagine you’re applying for a role that involves operating cloud services. An illustrative practice question might be:

Tell me about a production outage you helped investigate. How did you narrow down the cause, and how did you know the service had recovered?

If your answer focuses only on the fix, the coach could prompt you to explain your own role, the evidence you used, and the result. This is an example of the coaching interaction, not a captured model response.

Behind the chat, the specialists divide that work. They reach the document and record services through Model Context Protocol (MCP), a protocol for connecting agents to external capabilities.

Agent Job MCP tools
Receptionist Collect documents and set up the session MarkItDown and InterviewData
Behavioral Interviewer Ask about experience and give feedback InterviewData
Technical Interviewer Ask role-specific questions and discuss answers InterviewData
Summarizer Review the interview record and produce final feedback InterviewData
Triage Route the initial conversation and handle changes of direction None

MarkItDown converts documents into text the agents can use. InterviewData exposes operations for creating, retrieving, and updating interview records.

Following a resume through the application shows how these pieces work together.

Give Copilot interview tools, not a coding environment

An interview coach should be able to read a resume and save an interview record. It has no reason to run shell commands or edit the application’s source files.

The sample’s client configuration starts Copilot in CopilotClientMode.Empty. The agent factory then supplies the instructions and an explicit list of custom tools.

This excerpt from CreateCopilotSessionConfig shows that configuration. The full method also sets the model and permission handler:

var copilotTools = ToCopilotTools(tools);

return new SessionConfig
{
    AvailableTools = copilotTools
        .Select(tool => $"custom:{tool.Name}")
        .ToList(),
    SystemMessage = new SystemMessageConfig
    {
        Mode = SystemMessageMode.Append,
        Content = instructions,
    },
    Tools = copilotTools,
};

Tools supplies the custom definitions and callable handlers. AvailableTools identifies which tools the agent may use. The custom: prefix selects those supplied tools rather than Copilot CLI’s built-in tools.

When a candidate supplies a resume link, the Receptionist can use MarkItDown to convert the document. It then has InterviewData save the relevant context to the session record. That tool writes to Cosmos DB and returns a result the runtime can use in its next response. The agent doesn’t need a database client or built-in filesystem tools to perform these operations.

Tool selection controls exposure, but it is not a complete security boundary. The sample’s permission handler approves tool permission requests. A deployed application still needs authorization checks and an appropriate policy for the actions its tools can perform.

Connect Copilot to the interview workflow

The application uses Microsoft Agent Framework to connect the interview specialists. It represents each one as an AIAgent, which the workflow can invoke and hand control to.

The Copilot adapter connects the SDK runtime to that abstraction.

The agent factory creates a Copilot-backed specialist like this:

private static AIAgent CreateCopilotRunAgent(
    CopilotClient client,
    string name,
    string description,
    string? model,
    string instructions,
    IList<AITool>? tools)
{
    return client.AsAIAgent(
        CreateCopilotSessionConfig(model, instructions, tools),
        ownsClient: false,
        name: name,
        description: description);
}

AsAIAgent() lets the workflow use Copilot through the interface it already understands. ownsClient: false keeps the wrapper from taking ownership of the shared Copilot client, whose lifetime is managed by the application.

The responsibilities remain distinct. Copilot handles model interactions and tool execution for a specialist. Agent Framework connects the specialists and manages their handoffs. Application instructions describe the interview itself.

For example, the Behavioral Interviewer is instructed to use the STAR method: Situation, Task, Action, Result. It asks questions one at a time, gives feedback, and records the exchange. You can revise that coaching behavior without changing how the application communicates with Copilot.

This integration builds on the original Foundry-powered application. Agent Framework, specialist prompts, and MCP tools were already there. Adding Copilot did not require a second UI or another set of interview instructions: its adapter participates through the same AIAgent interface. Foundry remains supported. That reuse is a consequence of the design, rather than a new orchestration capability introduced by Copilot.

Include the tools that let specialists hand off

An agent needs tools for transferring control as well as tools for its interview work.

After collecting the documents, the Receptionist normally transfers the conversation to the Behavioral Interviewer. The Technical Interviewer and Summarizer follow. Triage is available when the candidate asks to change direction.

Agent Framework expresses those connections in the handoff builder:

var workflow = AgentWorkflowBuilder
    .CreateHandoffBuilderWith(triageAgent)
    .WithHandoffs(triageAgent, [receptionistAgent, behaviouralAgent, technicalAgent, summariserAgent])
    .WithHandoffs(receptionistAgent, [behaviouralAgent, triageAgent])
    .WithHandoffs(behaviouralAgent, [technicalAgent, triageAgent])
    .WithHandoffs(technicalAgent, [summariserAgent, triageAgent])
    .WithHandoff(summariserAgent, triageAgent)
    .Build();

The framework supplies transfer tools and additional instructions when it invokes an agent. The sample implementation at commit 68fd993 includes an adapter workaround because those run-time options were not automatically applied on the Copilot path. The repository uses floating package versions, so this describes the linked implementation rather than a limitation of every Copilot SDK release.

Supplying only the initial MCP tools leaves out the functions needed to hand off. The sample merges both sets before creating the Copilot agent for that invocation:

internal static IList<AITool> MergeCopilotTools(
    IList<AITool>? configuredTools,
    AgentRunOptions? options)
{
    var runTools = (options as ChatClientAgentRunOptions)?.ChatOptions?.Tools;

    return (configuredTools ?? [])
        .Concat(runTools ?? [])
        .DistinctBy(tool => tool.Name, StringComparer.Ordinal)
        .ToList();
}

The configured tools perform the interview work. The run-time tools allow transfers between specialists. Deduplicating by name avoids supplying the same tool twice.

MergeCopilotInstructions appends the run-time instructions to the specialist’s prompt. The integration also wraps handoff tool definitions as functions the SDK can call. It creates a lightweight agent wrapper for each invocation while reusing the Copilot client. This handling applies to both regular and streaming responses.

When integrating a runtime with an orchestration framework, check what the framework supplies at invocation time. Static tools alone may serve a single agent, but this interview also needs the tools that control its progression.

Running the sample

Local authentication can use the developer’s signed-in GitHub credentials. If you configure a token, the application uses it instead of relying on an existing sign-in. The .NET SDK bundles the Copilot CLI runtime it needs, and this configuration does not provision a Foundry model deployment.

The other services still need to run. Aspire starts the application components; containers run MarkItDown and the local Cosmos DB emulator. InterviewData now uses Cosmos DB rather than the earlier SQLite store, but the agents continue to access records through MCP.

For developers retaining the Foundry configuration, Aspire now provisions the resource and model deployment without a separate provisioning step. Azure access uses DefaultAzureCredential rather than a Foundry API key. The provider guides cover that path as well as Copilot authentication, deployment, and configuration.

SDK requests count against the authenticated account’s Copilot usage. Available models and access depend on the plan and organization policy. Existing access does not mean unlimited or automatically free requests, and it does not supply an authentication design for a public application with many users.

Use sample candidate data. Although Cosmos DB stores interview records, uploaded document bytes remain in memory, and the sample does not implement complete workflow recovery. It also exposes DevUI outside development. Before handling real resumes, restrict access to the application and developer tools, review tool permissions, and establish data-retention policies.

Put Copilot to work in your own app

Interview Coach gives Copilot a job outside the editor: work with a candidate’s documents and answers, call the application’s tools, and help produce interview feedback. The SDK supplies the model and tool-call loop. Agent Framework connects the specialists, and the application defines the coaching behavior.

Try the application using the Copilot setup guide. After an interview with sample data, change the feedback instructions for one specialist or connect a tool for your own workflow. You can experiment with the experience you want to build while Copilot handles the runtime interactions underneath it.

Further reading

The post Build an interview coach app with the GitHub Copilot SDK appeared first on Microsoft for Developers.

Read the whole story
alvinashcraft
20 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Tell AI Everything Twice, Then Tell It a Second Time

1 Share
Imagine that you sit down with [Claude Code](https://claude.com/product/claude-code) or [Cursor](https://cursor.com) or [Copilot](https://github.com/features/copilot) to write a careful prompt: ```text Build an API that is multi-tenant. Every query must be scoped to the tenant ID grabbed from the JWT. Never, ever trust a tenant ID that comes in on the request body or query string. ``` You watch it scaffold the first few endpoints. It does exactly what you asked. The tenant ID comes off the JWT claims and gets passed into your repository calls. Feeling confident, you go get some coffee and come back fifteen minutes later. It's on endpoint number eleven now, and you watch it reading `tenantId` straight out of the query string. "What in the world?!" You didn't tell it to stop using the JWT. Is it stupid, or is it trying to sabotage you? ## AI: The Dumbest Genius If you tell AI to create a to-do app and sit back for a few minutes, it can create a flawless one 10 times as fast as you could have even typed in the code for one yourself. Enough experiences like this and you get lulled into a false sense of security. You start to feel like your AI assistant knows all. Imagine that you hired a developer, told them "the single most important rule on this project is tenant isolation," walked out of the room, and came back a minute later to find them writing an endpoint that lets any customer read any other customer's data. You'd assume something was wrong with them. For all our flaws, no human is going to forget a critical thing in just sixty seconds. And because we like to anthropomorphize our LLMs, it catches us by surprise when they do something this stupid. > **An LLM isn't remembering your requirements. It's predicting the next token, and your requirements are just one of many inputs that are tugging on that prediction.** When your instruction was thirty seconds ago, it still has a lot of pull over what your LLM does. Thirty minutes later, buried in the context under file contents and tool output and the LLM's own chatter, it has much less. By the time the LLM gets to implementing endpoint number eleven, it asks itself "what usually comes next here?", and the answer from the millions of examples it saw in training is "a tenantId parameter from the query string." Your one critical instruction is fading into the background. ## Where I've Watched It Forget Below are three places just recently where I gave the model a clear rule up front and it drifted anyway: ### Security I asked for a "secure" app and got one…mostly. It built out parameterized queries, used input validation on the DTOs, everything you would want! But then I ran a security scan after it was done (something like [Semgrep](https://semgrep.dev), [Snyk](https://snyk.io), or GitHub's [CodeQL](https://codeql.github.com)) and it flagged a raw `FromSqlRaw` with a string-interpolated `WHERE` clause. (For more on scanning AI-written code, see our post [Locking Down AI: Strategies for Uncovering Vulnerabilities](/locking-down-ai-strategies-for-uncovering-vulnerabilities/).) ### Architecture On a modernization project I told the model up front to use clean architecture, and that the Domain project should have no reference to EF Core because all persistence goes through repository interfaces. It set up the projects perfectly and wrote a nice little README explaining the rule, but about halfway through implementing the feature slices, a handler in the Application layer started newing up a `DbContext` directly because that was, frankly, the shortest path to making the test pass. ### Coding standards "Use our `Result` type for failures, don't throw exceptions for expected error cases." It followed the rule for the first six methods. Method seven, however, throws an `InvalidOperationException`. Method eight is back to `Result`. It's not even consistent about being wrong! If this is how these LLMs tend to work, then how do we write quality code with them? Better prompting isn't the fix, and neither is ALL CAPS. The answer is check gates. ## Requirements First, Check Gates After Here's the mental model that finally got me out of the "why do I have to keep saying this" loop: > **If something is important enough to tell the AI up front, it's important enough to check for afterward. The instruction is a requirement. The check is a separate gate that comes after.** On my projects, that looks like: ### 1. Put the rules where they get re-read Keep a [`CLAUDE.md`](https://academy.claude.com/courses/claude-code-101/the-claude-md-file) or [`AGENTS.md`](https://agents.md) at the repo root with your most important rules. These get pulled back into context every time you start a new session. Inside a long session it can still drift, but this resets what's critical every time you start fresh. ### 2. Turn the rules into tests An architecture test ([ArchUnitNET](https://archunitnet.readthedocs.io), or [NetArchTest](https://github.com/BenMorris/NetArchTest) if you're on the older stuff) that asserts the Domain project doesn't reference `Microsoft.EntityFrameworkCore` does more for you than a paragraph of prose about clean architecture. A [Roslyn analyzer](https://learn.microsoft.com/dotnet/fundamentals/code-analysis/overview) that flags `FromSqlRaw` is better than the word "secure" in a prompt. If the model can break the rule silently, it sometimes will. If breaking the rule turns the build red, the model will fix it. ### 3. Run scans Security scan, dependency audit, whatever your equivalent is. Put it in your CI/CD process or tell the agent to run it itself and fix what it finds. In my experience, it's very good at fixing findings you feed to it. ### 4. Have a second model do a code review Ask a fresh session (or a different model entirely, [Codex](https://openai.com/codex/) reviewing Claude Code's output or vice versa) to review the diff against the original requirements. Give it the rules and the changes and nothing else. It has no memory of the shortcuts the first model took and no investment in them, and it catches things the original session would swear it didn't do. It's code review by someone who wasn't the developer, which is the whole point of code review. None of those are new ideas. We've had CI gates and code review and static analysis for over twenty years. What's new is realizing you need to apply them to LLMs as well as humans. And as I wrote in [Developing with AI Exposes Your Bottlenecks](/developing-with-ai-exposes-your-bottlenecks/), the gates are where the work piles up once coding gets fast, so they're worth investing in. ## Where the Analogy Breaks I've been talking about LLMs as a forgetful genius developer, and that's a useful analogy, but not perfect. After all, a human who forgets something will usually feel bad enough about it to not do it again. Your LLM might not make the same mistake *in the same session* after you correct it, but then it'll happily do it again tomorrow with another "Good catch!" when you point it out. It doesn't learn like that, so you need to set up your processes to catch its weaknesses. If you want help building a solid process around your AI-assisted development, [reach out to us at Trailhead](/contact-us/). We have a lot of experience building gates into our projects, and we'd be happy to help you tighten up your AI development processes.

The post Tell AI Everything Twice, Then Tell It a Second Time appeared first on Trailhead Technology Partners.

Read the whole story
alvinashcraft
31 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Things That Caught My Attention Last Week - September 14

1 Share

caught-my-i

Software Architecture

We all wanted to be Netflix, who do we want to be next? by Oskar Dudycz

.NET

XPath for custom types in .NET by Gérald Barré

PostgreSQL Row-Level Security With EF Core and Npgsql by Milan Jovanović

Today I will... find hidden latency across a distributed .NET application by Visual Studio Blog

Use C# unions and closed hierarchies in ASP.NET Core by .NET Team

.NET and .NET Framework September 2026 servicing releases updates by .NET Team

Announcing .NET 11 Release Candidate 1 by .NET Team

Worse is better: C# versus F# by Mark Seemann

REST/APIs

Agent Resource Discovery (ARD): Sixty-Two Companies Declared An OpenAPI. They Spelled It Eleven Ways. by Kin Lane

Mintlify Does A Lot With One OpenAPI Extension by Kin Lane

Microsoft Works In The Open With 70,000 Engineers. You Can Publish Your API Documentation. by Kin Lane

Azure

September Patches for Azure DevOps Server by Azure DevOps Blog

MulticloudDB SDK: Cross-cloud portability in the coding agent era - Azure Cosmos DB Blog by Azure Cosmos DB Blog

Software Development

Architecture and model diffs via code conventions by Nick Tune

GitHub availability report: August 2026 by The GitHub Blog

AI

GitHub Copilot app for Beginners: Using the diff, terminal, and browser by The GitHub Blog

Agent Skills: New Value, New Problems by Kin Lane

Read the whole story
alvinashcraft
49 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Patching in 2026 with Susan Bradley

1 Share

How has patching changed in 2026? Richard chats with patching goddess Susan Bradley about what she's seeing in vendor patches and how they're affecting consumers and businesses. On one hand, you have the security risk of unpatched servers, which puts a lot of pressure on sysadmins to deploy patches immediately. And, at the same time, the number of patches has grown massively as tools like Anthropic's Fable have revealed far more vulnerabilities. Then there are the vendors that are slow to get fixes out for drivers and firmware for existing systems, increasing the risk of serious failures - it's not easy patching in 2026!

Links

Recorded Aug 21, 2026





Download audio: https://cdn.simplecast.com/media/audio/transcoded/5379899c-61c5-43c3-aa3f-1128cffd9ef4/c2165e35-09c6-4ae8-b29e-2d26dad5aece/episodes/audio/group/60e1970e-bf0d-4337-863a-1041053076b5/group-item/ed9c2510-851f-4750-b3a0-555d81240d4a/128_default_tc.mp3?aid=rss_feed&feed=cRTTfxcT
Read the whole story
alvinashcraft
7 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

MBW 1042: Tassels Galore! - iOS 27, iPadOS 27, and More Out Now!

1 Share

Apple held its first Apple event with John Ternus as CEO, and unveiled the iPhone Duo! Apple announced the Apple Watch Series 12 and Ultra 4 with a new AI feature called Siri Recap, bringing some concerns over its continuous ambient listening. And Apple wins 29 Emmys at the 78th Primetime Emmy Awards!

  • John Ternus enters, with folding phones and AI watches.
  • Apple unveils the iPhone Duo, a foldable phone that costs $1,999.
  • Apple Watch Series 12 and Ultra 4 unveiled with upgraded health tracking system.
  • Apple's always-listening watch features test eavesdropping laws.
  • Apple's OS 27 updates will be released on Monday, September 14th.
  • Apple hid a classic Mac easter egg in the iOS 27 Settings app.
  • pdfu on X: "iOS 27 and macOS Golden Gate have private hooks that let apps add Siri Extensions..."
  • Apple's Siri AI can be swapped out for Claude, ChatGPT, code shows.
  • macOS 27 Golden Gate review: Bridging the Tahoe gap.
  • visionOS 27 updates.
  • Apple wins most Emmys.

Picks of the Week

  • Christina's Picks: Spigen iPhone Duo Cast Concept & View Tweet extensions.
  • Andy's Pick: Mac Duo
  • Jason's Pick: Strategery 4
  • Leo's Pick: Homebrew 7

Hosts: Leo Laporte, Andy Ihnatko, Jason Snell, and Christina Warren

Download or subscribe to MacBreak Weekly at https://twit.tv/shows/macbreak-weekly.

Join Club TWiT for Ad-Free Podcasts!
Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit

Sponsors:





Download audio: https://pdst.fm/e/pscrb.fm/rss/p/mgln.ai/e/294/cdn.twit.tv/megaphone/mbw_1042/ARML6040453802.mp3
Read the whole story
alvinashcraft
7 minutes ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories