When a test fails in CI, “the build is red” is only the starting point. The useful questions are whether this change caused the failure, whether the test has failed before, and where to find the evidence. If answering them means searching a job log, downloading an artifact, and finding the teammate who remembers how that test behaved last month, the report has not done enough.

MTP reporting shortens the path from a failed build to the decision a developer or reviewer needs to make.

Microsoft.Testing.Platform (MTP) powers dotnet test, the Test Explorer in Visual Studio and Visual Studio Code, and test runs in CI. Over the past year its reporting has been closing that gap — putting failures where reviewers already work, and preserving the evidence when a run falls over.

This works on whichever test framework you already use: MTP is supported by MSTest, NUnit, xUnit.net, TUnit and Expecto. Most capabilities in this post require MTP 2.3.0 or later. The examples were validated with MSTest.Sdk 4.3.3 and MTP 2.3.3; the GitHub Actions, JUnit and CTRF reporters currently ship as preview packages. Most of what follows is provider-independent, but two of the strongest features read your build history, and that part is Azure DevOps only today:

Start with inline reporting on either provider: it moves the next failure out of the raw log. If you use Azure DevOps, add history-based triage next so reviewers can distinguish a new regression from a recurring failure.

Put the failure where the author is already looking

A report that lands in an artifact is a report someone has to go and fetch. Both providers can surface results inline instead, and MTP now emits the annotation format each one understands natively.

Enable --report-gh in a GitHub Actions job and a failing test becomes an annotation on the line of code that failed. Skipped tests appear as warnings, each assembly is collapsed into its own log group, and the same run writes a summary straight to the workflow page:

The failure is visible from the job page and the pull request without downloading a report artifact. Annotations, log groups, the summary and slow-test notices can each be configured independently.

Azure DevOps has the same shape behind --report-azdo, and it offers something GitHub Actions doesn’t have yet. With --publish-azdo-test-results, results stream into the Tests tab while the run is still going — instead of arriving from a separate publish step that only runs if the job survives to reach it. The two are independent: --report-azdo handles annotations and logging, --publish-azdo-test-results handles the Tests tab, and either works alone.

Azure DevOps: tell a genuine regression from a known flake

This is the central decision that build history unlocks: not only which test failed, but whether the current change is likely to have caused it.

In a large suite, intermittently failing tests train reviewers to read red as background noise. Once “probably just flaky” is the reasonable first guess, real regressions inherit the same shrug — and the cost isn’t the flaky tests, it’s the response time on everything else.

Azure DevOps already exposes this evidence in the Tests tab. In this testfx run, the current build reported no failures across 86,406 tests, while the 14-day history still called out one unique failing test:

For the testfx team, that context narrows the first investigation step: a failure with no history gets immediate regression attention, while a recurring failure starts with its existing flaky record.

Pass that same 14-day window to the reporter, and it queries the pipeline history and attaches the context to each failure:

dotnet test --report-azdo --report-azdo-flaky-history 14

A test that has been failing on and off for two weeks is labelled with its record:

[flaky: failed 3/20 in last 14d]

A test with no such history gets the label that matters:

[REGRESSION]

That turns “someone should look at this eventually” into “this pull request broke something,” and it reaches the reviewer at the point of decision instead of depending on who remembers what. If you want CI to act on it rather than merely say it, --report-azdo-demote-known-flaky turns known-flaky failures into warnings while regressions stay errors.

In the testfx pipeline we use the history annotations but leave automatic demotion off, so every failure stays blocking and history only guides triage. Other teams will reasonably choose to demote. The decision worth making explicitly is whether history should inform a reviewer or change CI severity on its own.

The same history powers slow-test detection. Instead of one threshold that is wrong for every project, --report-azdo-slow-test-history compares each test against its own past. The threshold is a configurable multiple over a minimum number of runs, so a single cold start won’t trip it.

Keep the evidence when the run crashes

History helps determine whether a failure is new, but triage still stalls if the run loses the evidence needed to investigate it. The run you most need a report from is the one that died. That used to be the run that produced nothing, because results were serialized at the end.

TRX results are now streamed to disk as they are produced, so a hard crash no longer takes the whole report with it. Pair --report-trx with the crash-dump extension and the run is supervised by a controller process that finalises the partial report when the host dies:

dotnet test --report-trx --crashdump

You get a valid TRX containing every test that completed, plus a console list of the ones that didn’t:

The following tests were still running when the test host crashed:
[00:00:00] D_crash_the_host

That names the test that took the process down, without rerunning the suite to find it. The extension also writes a *.crash.sequence.log recording every test start and end, so a test that started and never finished is unambiguous even when several were running in parallel.

Attachments got the same treatment. Crash dumps, hang dumps and extension artifacts are no longer silently dropped on .NET Framework when the path exceeds the Windows MAX_PATH limit, and an attachment that can’t be copied is now surfaced on the console rather than only inside the TRX file. An incomplete run is now visibly incomplete, instead of a green-looking report with the evidence quietly missing.

One run, reports for people and tools

Choose the output based on who consumes it. Use TRX for .NET tooling, HTML for direct inspection, and JUnit or CTRF for dashboards and cross-stack automation. A single run can enable any combination, removing the format-conversion step teams often wire into their pipelines.

CTRF is worth knowing about if you aggregate results across languages: it’s a shared JSON schema, so .NET reports into the same shape as the rest of a polyglot estate.

Only TRX and JUnit are formats a CI system parses into a results view. HTML and CTRF are for people and dashboards, so they show up as downloadable artifacts instead — and on Azure DevOps, --report-azdo-upload-artifacts files collects them automatically. Pick TRX or JUnit when something downstream reads the results; add HTML or CTRF for whoever has to look at them.

Report names no longer collide either. Each reporter takes a --report-<format>-filename accepting build-specific placeholders such as {asm} and {tfm}, resolved under the results directory:

dotnet test --report-trx --report-trx-filename "reports/{asm}_{tfm}_{time}.trx"

Left alone, TRX, HTML and JUnit now default to a deterministic <asm>_<tfm>_<arch> name. If you multi-target, net8.0 and net8.0-windows write separate files instead of reporting over each other — a quiet data-loss bug in matrix builds that is worth checking for in your current setup.

Stable output for automation

People need actionable context in the pull request; automation needs the same context in a stable, structured format. Scripts, dashboards, IDE integrations and coding agents cannot reliably consume terminal prose.

--list-tests json emits a schema-versioned document describing every discovered test, down to its source location:

{
  "schemaVersion": 1,
  "tests": [
    {
      "uid": "11a3aade-7a31-8389-89ce-eab6ff0db7f3",
      "displayName": "Total_includes_shipping",
      "location": { "file": "CheckoutTests.cs", "lineStart": 13, "lineEnd": 13 }
    }
  ]
}

That schema is a stable input for test selection, impact analysis or IDE integration, instead of scraping console text that changes between releases.

MTP also adapts its own output for these consumers. In an agent or LLM environment it suppresses the banner, ANSI escapes and progress animation, and defaults --show-stdout and --show-stderr to failed, so what comes back is results rather than redrawn progress bars. You can set the same behaviour by hand — the platform honors NO_COLOR, and exposes --ansi and --progress for the rest.

Try it today

1. Confirm that one test project runs MTP 2.3 or later.
2. Enable the reporter for your CI provider through MSTest.Sdk or a direct package reference.
3. Add dotnet test --report-gh in GitHub Actions or dotnet test --report-azdo in Azure DevOps.
4. Use the next failure to check whether the report reduced the time you spent searching logs.

Scale it across a repository

Once you have chosen a reporting policy, keep it in the repository rather than everyone’s shell history. Every option in the MTP CLI reference can live in a checked-in testconfig.json beside your test project, under commandLineOptions:

{
  "commandLineOptions": {
    "report-trx": true,
    "report-html": true,
    "report-azdo": true,
    "report-azdo-flaky-history": 14
  }
}

Local and CI runs then produce the same reports by default, and a developer can still override any of it from the command line. Misspelled settings fail at startup with a message naming the file, rather than being silently ignored:

In testconfig.json under 'commandLineOptions': Unknown option '--report-trxx'

For a repo-wide rollout:

• Centralize once you’ve decided. With MSTest.Sdk each reporter is an MSBuild property, so one Directory.Build.props applies your policy to every test project in the repo:

  <Project>
  <PropertyGroup>
    <EnableMicrosoftTestingExtensionsHtmlReport>true</EnableMicrosoftTestingExtensionsHtmlReport>
    <EnableMicrosoftTestingExtensionsAzureDevOpsReport>true</EnableMicrosoftTestingExtensionsAzureDevOpsReport>
  </PropertyGroup>
  </Project>

  The properties follow one pattern (EnableMicrosoftTestingExtensions + reporter name), and <TestingExtensionsProfile>AllMicrosoft</TestingExtensionsProfile> switches on the stable set in one line — TRX, HTML, Azure DevOps, GitHub Actions, crash dump, hang dump and retry. JUnit and CTRF stay opt-in.

• On other frameworks, check the version. Outside MSTest.Sdk you reference the reporter package directly, and it has to match the MTP version your framework targets. These reporters are built on MTP 2.x, so adding one to a project on an older framework release fails at startup with a MissingMethodException. MTP 2.x support arrives in MSTest.TestAdapter 4.0.0, NUnit3TestAdapter 6.0.1, TUnit 1.7.16, YoloDev.Expecto.TestSdk 0.16.0, and the xunit.v3 4.0 prereleases.
• Set the runner opt-in once, at the repo level. Whether dotnet test runs your solution through MTP is a single switch — put it in Directory.Build.props so a new project can’t drift out of it. Mixing MTP and VSTest projects in one solution isn’t supported; on the .NET 10 SDK, running an MTP project through the old VSTest path fails the build with a link to the opt-in, so any drift surfaces immediately. The migration guide covers that switch.
• The Azure DevOps history options need a token. --report-azdo-flaky-history and --report-azdo-slow-test-history call the Azure DevOps REST API, so pass SYSTEM_ACCESSTOKEN: $(System.AccessToken) to the step. Without it the run continues and simply skips the history annotations.

Replacing your existing publish tasks

If you own an existing Azure DevOps pipeline, this section shows which publishing tasks you can replace. Live publishing sends the results it already holds in memory straight to the REST API — it doesn’t read a TRX, so there’s no report file to produce or glob for. Together with --report-azdo-upload-artifacts files, which publishes the test results directory as a build artifact, that replaces the tasks most pipelines carry today:

Coverage is the one to watch. Live publishing does attach .coverage, .cobertura.xml and .opencover.xml to the test run, and failing tests carry their own attachments — dumps, plus captured stdout and stderr. But nothing here populates the pipeline’s Code Coverage tab, so keep the step that does that today.

Where reporting goes next

Line these changes up and a direction appears. Two years ago MTP was a lighter way to execute tests. Today it puts failures directly into GitHub Actions and Azure DevOps, and publishes results live in Azure DevOps as the run happens. It interprets a failure against your own build history, preserves evidence when the host crashes, and exposes a stable schema to tools that hadn’t been written when the platform shipped. The next round — live publishing for GitHub Actions, richer artifact handling, and stable releases for the preview reporters — continues the same shift: from producing a result file to helping a team decide what to do next.

Reporting is a practical place to evaluate MTP, because developers, reviewers and build owners all see the difference on day one. Pick the smallest version of it: turn on --report-gh or --report-azdo in one test project and watch where your next failure shows up.

Then tell us what still sends you back to the logs. Several of the options in this post exist because somebody opened an issue on the testfx repository describing a bad afternoon.

The post Test reporting in Microsoft.Testing.Platform: from red build to root cause appeared first on .NET Blog.

If you have ever deployed a SQL Database Project (.sqlproj) or a .dacpac with SqlPackage, you know the drill: you kick off a publish against an empty or near-empty database, grab a coffee, and wait. Even for tiny schemas, every single publish reliably took 20+ seconds before anything actually happened on the server.

That fixed "startup tax" has been one of the most requested pain points for years - and I am thrilled to say it is finally being addressed. With the DacFx 170.5.60-preview release, a warm publish can now complete in around 2 seconds. That is a 10x improvement on the part of the deployment that used to feel like pure overhead.

I actually attempted something similar a while ago, and will share my experience from that attempt with the DacFX team.

Why was it always 20 seconds?

Every publish begins by loading and validating the model contained in the .dacpac, building an in-memory representation of your schema, and comparing it against the target, requiring large metadata queries against the target database. Historically a large chunk of that work was repeated on every run, regardless of whether anything had actually changed. For small projects the model work completely dominated the wall-clock time - hence the constant ~20 second floor.

The work latest sqlpackage and DacFX preview removes that redundant work, so repeated publishes of the same .dacpac no longer pay the full model cost each time.

How to try it out

The feature is in preview, so you need the latest preview build of SqlPackage.

1. Make sure you have the .NET SDK installed.

2. Install (or update to) the latest preview of the SqlPackage global tool:

   dotnet tool install -g microsoft.sqlpackage --prerelease
   

3. Confirm you are on the preview build (170.5.60.2 or higher):

   sqlpackage /version
   

4. Publish with the new additional parameters and enjoy the speed

   sqlpackage /Action:Publish ^
       /SourceFile:MyDatabase.dacpac ^
       /TargetServerName:localhost ^
       /TargetDatabaseName:MyDatabase ^
       /p:LogDeployment=true ^
       /p:EnableFastComparison=true
   

Run the same publish a second time to see the warm-path improvement in action.

You will see this logged to the console when fast deployment kicks in:

Fast comparison match found, source model checksum and deployment options match last logged deployment. Database reverse engineering and deployment plan generation is skipped. Disable fast comparison to run a full model comparison.

Notice that this feature assumes no drift (ouside schema changes) between deployments. In other words, it assumes that the .dacpac owns the database schema.

Under the hood

The key change is that the model checksum and additional metadata is now cached in a system table in the target database, so that subsequent publishes can skip the model reverse engineer and model build steps, resulting in significantly faster deployments.

The system table to store the "vectors" of the currently deployed package has the folloiwng structure:

CREATE TABLE [dbo].[__DacFxDeploymentHistory](
   [deployment_id] [int] IDENTITY(1,1) NOT NULL,
   [dacpac_name] [varchar](256) NOT NULL,
   [project_id] [uniqueidentifier] NOT NULL,
   [dac_version] [nvarchar](64) NOT NULL,
   [deployment_options] [xml] NULL,
   [model_checksum] [varbinary](32) NOT NULL,
   [date_created] [datetime2](7) NOT NULL,
   [created_by] [sysname] NOT NULL,
   [description] [nvarchar](4000) NULL,
 CONSTRAINT [PK___DacFxDeploymentHistory] PRIMARY KEY CLUSTERED 
(
  [deployment_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

ALTER TABLE [dbo].[__DacFxDeploymentHistory] ADD  DEFAULT (sysutcdatetime()) FOR [date_created]
GO

ALTER TABLE [dbo].[__DacFxDeploymentHistory] ADD  DEFAULT (original_login()) FOR [created_by]
GO

The /p:LogDeployment=true parameter is required to enable the creation of this table and the logging of the model checksum and deployment options etc.

You can monitor the contents of this table to see the last deployed model checksum and deployment options. You can also use this to verify that the fast comparison would work as expected before your turn on /p:EnableFastComparison=true.

Any changes to the schema will update the stored model checksum, so that subsequent publishes can detect that a full model comparison is required.

Likewise any changes to the deployment options will also update the stored deployment options, so that subsequent publishes can detect that a full model comparison is required.

Based on this query executed by sqlpackage, you can see that the decision to use skip full deployment is based on entries scoped to the current ProjectGuid of the .dacpac:

exec sp_executesql N'SELECT TOP 1 
      [deployment_id], [dacpac_name], [project_id], [dac_version],
      [deployment_options], [model_checksum],
      [date_created], [created_by], [description]
FROM [dbo].[__DacFxDeploymentHistory] WITH (NOLOCK)
WHERE [project_id] = @projectId
ORDER BY [date_created] DESC',N'@projectId uniqueidentifier',@projectId='726D17BA-DCCA-4607-8C1F-3A68523C7112'

/p:EnableFastComparison=true requires /p:LogDeployment=true in order to work. And if the outcome of the fast comparision is a match, then deployment is skipped.

I tried changing a post deployment script, and the next publish incorrectly skipped the model comparison, so it seems that only schema and deployment options changes are detected.

This is a known limitation of the current implementation. I have filed a bug for this.

One caveat: the missing ProjectGuid

While trying this out, you may hit an issue where the speed-up does not kick in because the .dacpac is missing a ProjectGuid, which is used to identify the project across publishes. This is being tracked in microsoft/DacFx#824. If you do not see the improved timings, this is the most likely culprit - keep an eye on the issue for the recommended workaround and follow-up fixes.

Wrapping up

This is a preview, so expect a few rough edges, but it is a fantastic step forward for anyone doing frequent, iterative database deployments - especially in inner-loop development and CI where those seconds add up fast.

Give it a try, and please report any feedback on the DacFx repo. Huge thanks to the DacFx team for tackling this long-standing request!

Happy (fast) deploying!

Labor Day is creeping up so fast, the ink’s barely dry on LeBron James’ contract with the Philadelphia 76ers. (Can you tell we’re still not over it?)

So, yes, it’s time to get out there, wear your pre-Labor Day white and experience summer before it’s gone. 

From seeing Kyle Schwarber swat a home run with the Philadelphia Phillies to observing the Bian Lian face-changers at the Philadelphia Chinese Lantern Festival, August is the perfect month to check off all the things you forgot to do this summer — with the pomp and circumstance of America’s 250th birthday celebration still setting a patriotic soundtrack.

Enjoy beers and bears during the Summer Ale Festival at Philadelphia Zoo, cool off with fun on the water at Spruce Street Harbor Park, get a heaping helping of history with new exhibitions at sites like the National Liberty Museum or the Museum of the American Revolution, and so much more.

Read on for some of the best things to do in Philly before summer’s over.

This is a preview build of WinGet for those interested in trying out upcoming features and fixes. While it has had some use and should be free of major issues, it may have bugs or usability problems. If you find any, please help us out by filing an issue.

New in v1.30

Nothing yet.

Bug Fixes

• Updated NUnit to v4
• Fixed a crash (0x8000ffff) when using --disable-interactivity with the Resume experimental feature enabled during install operations.

What's Changed

• Bubble up child process errors in IndexCreationTool by Tom Plant (@pl4nty) in #6261
• Bump brace-expansion and @vscode/vsce in /tools/WinGetLogViewer by Dependabot (@dependabot)[bot] in #6380
• Bump manifest version to 1.30 by Kaleb Luedtke (@Trenly) in #6379
• Bump linkify-it from 5.0.1 to 5.0.2 in /tools/WinGetLogViewer by Dependabot (@dependabot)[bot] in #6384
• Bump fast-uri from 3.1.2 to 3.1.4 in /tools/WinGetLogViewer by Dependabot (@dependabot)[bot] in #6385
• Block admin uninstall of user package by JohnMcPMS in #6345

Full Changelog: v1.30.80-preview...v1.30.90-preview

Simon Willison on Technical Blogging

It includes my answers to the following questions:

• Why did you start blogging – and why do you continue?
• What has been the most surprising impact of blogging for you?
• What blog post are you most proud of and why?
• What post was the most difficult to write and how did you tackle it?
• Any lessons learned that you want to share with the community?
• Your advice for people just getting started with blogging?
• A few blogs that you particularly enjoy?

I'll repeat my most important piece of advice here:

My number one tip for blogging is to lower your standards! Aim to hit publish while you are still actively unhappy with what you have written, because the only alternative is a huge folder full of drafts and never publishing anything at all.

Nobody will ever know how perfect the thing you intended to write would have been. The flaws you see in your writing are invisible to everyone else.

Tags: blogging, interviews