Avoid these common infrastructure-related mistakes when you’re ready to scale your .NET app to keep your performance at scale too.

IT teams often assume that scaling a .NET application means moving it to a larger virtual machine or containerizing the application. While such actions do have their place, performance problems are often tied to infrastructure design rather than the application itself.

This article discusses some of the most common infrastructure-related mistakes that are often made when scaling a .NET application.

1. Adding Instances Without Thinking About Traffic Distribution

Application scaling very often involves adding additional nodes to handle the increased demand. However, it is a mistake to add nodes without considering how requests will reach those nodes. Oftentimes organizations will use round-robin load balancing to evenly distribute inbound requests across the available nodes. However, load balancing can break down when session affinity comes into play.

Session affinity occurs as a result of session data being stored on a specific server. This in turn causes all of a user’s requests to be sent to that same server, because that’s where the user’s session lives. This isn’t necessarily a problem by itself, but session affinity can become problematic under the right conditions, especially when there is a mixture of active and inactive users.

In such situations, a particular instance may become overloaded, even if plenty of capacity exists on other instances, simply because the session is bound to that instance. Additionally, if that instance were to fail, then the users on that instance lose their session.

A better approach is to design services to be stateless and to store session data within shared storage rather than placing it in an instance’s memory. This allows the load balancer to freely distribute requests.

2. Assuming That All Requests Look Alike

Basic load distribution strategies, such as round robin, assume that every instance can deliver the same level of performance and that every request has the same impact. In reality though, these assumptions are naïve.

Some requests, for example, trigger longer running operations, call external services or make more complex database queries. As such, some requests will inevitably be completed more quickly than others. This can result in some nodes being bogged down while others remain nearly idle.

The best way to avoid this type of problem is to avoid blindly distributing traffic across nodes. Instead, consider each node’s health when making routing decisions. As an example, if a node were experiencing excessive latency or had a significant queue depth, then it would be better to route inbound requests to a node that is under less of a load rather than assigning the request to the struggling node just because it is that node’s turn.

3. Not Having a Load-Shedding Strategy

Although it may seem counterintuitive, load shedding can sometimes be the most appropriate course of action. For those who might not be familiar with the term, load shedding refers to the practice of intentionally rejecting certain requests as a way of helping the system to stay healthy.

To use an analogy from everyday life, consider what might happen if a restaurant kept seating additional guests in spite of the kitchen and the waitstaff being completely overwhelmed. In such a situation, everyone would receive poor service. Conversely, if the restaurant were to pause and not seat anyone else until the staff had a chance to catch up, then it would result in a better overall experience for everyone.

Load shedding works similarly. When a system becomes overwhelmed with more requests than the nodes can handle, then the load balancer intentionally fails some requests so that those requests that do make it through can still be serviced normally.

4. Lack of Visibility

This one seems so simple (and it is), yet it is so important. Organizations often make the mistake of just assuming that load balancing is working as intended without doing anything to verify their assumptions. The problem is that if an organization cannot see how requests are being routed, then scaling an application becomes guesswork.

Without proper visibility, you can’t tell whether traffic is being evenly distributed, if there are nodes that are constantly struggling or which instances are handling slow requests. It becomes easy to misdiagnose bottlenecks and blame the application for poor performance when routing is the real problem (or vice versa).

Ideally, an organization should be able to collect per-instance metrics from its nodes including CPU and memory usage and the request count. There should also be a way to trace requests as they flow across services and there should be a way to track load distribution metrics to see how the individual nodes are being leveraged.

5. Treating Load Balancing as an Afterthought

One of the single biggest mistakes that is often made when scaling an application is treating load balancing as an afterthought. Typically, an application is deployed and works fine. Eventually though, demand increases and the application begins to struggle. IT distributes the application across several nodes and places a basic load balancer in front of those nodes. In these types of situations, load balancers often use default settings because, “round robin is good enough for not” or “the cloud provider’s defaults should be fine.”

The end result is that while the application has indeed been scaled, there is a good chance that some of the original problems still exist. The added infrastructure now makes those problems more difficult to diagnose.

True load balancing is not about spreading traffic evenly. It’s about making informed decisions as to where each inbound request should be routed. This informed decision making means using application aware routing to base routing decisions on application behavior, not just routing rules. It also means taking instance health and readiness into account and continually adjusting routing decisions based on latency, errors and available capacity.

Modern applications are evolving rapidly, with growing demand for AI-powered experiences, interactive data visualization, and productivity-focused user interfaces. To help you meet these expectations, the Essential Studio^® 2026 Volume 2 release for .NET MAUI introduces enhancements across AI experiences, data management, and visualization.

This release brings new capabilities that make it easier to build intelligent applications, work with data more effectively, and deliver richer user experiences. From improved AI interactions and conversation management to enhanced charting and productivity-focused data features, these updates help you create modern cross-platform applications with less effort.

Let’s explore the highlights of the 2026 Volume 2 release.

AI AssistView

AI-powered applications continue to evolve, and this release expands AI AssistView with features that make it easier than ever to build sophisticated conversational experiences.

Agent picker

Today’s applications often rely on multiple AI agents, each tailored for a specific task such as document summarization, coding assistance, customer support, or translation. With the new Agent Picker, users can switch between agents directly from the prompt editor without interrupting their workflow.

The selected agent appears as a compact chip with an icon and label, making it easy to identify the active assistant at a glance.

Conversation management

As AI applications grow, managing conversations becomes just as important as generating responses. The new Conversation Management support enables developers to provide a dedicated conversation panel where users can:

• Pin important conversations
• Rename chats for easier identification
• Filter conversations
• Remove conversations that are no longer needed

This creates a familiar experience similar to modern AI chat platforms while keeping conversation history organized and accessible.

AI chat experience enhancements

Several usability improvements have also been added to make AI interactions more seamless:

• Image Preview allows users to open attached images in a larger view for easier inspection of screenshots, diagrams, and other visual content.
  

  

• Time-based Message Grouping automatically organizes conversations into sections such as Today, Yesterday, and earlier dates, making long chat histories easier to navigate.
  

  

• Resizable Prompt Input provides additional editing space for longer prompts, detailed instructions, and code snippets.
  

  

Together, these enhancements create a more natural and productive chat experience without requiring additional customization.

Cartesian Charts

The Cartesian Charts control gains new visualization capabilities that help developers present complex information more effectively.

Point color mapping

Data often needs visual differentiation to highlight categories, trends, or exceptions. Point Color Mapping allows colors to be assigned directly from the data source using brushes, color values, or hexadecimal color codes.

This makes it simple to create more expressive charts while keeping configuration straightforward.

Multilevel labels

Large datasets frequently contain multiple layers of categorization. Multilevel Labels help organize axis labels into hierarchical groups, making complex information easier to interpret.

This enhancement is especially useful for dashboards, scheduling applications, financial reports, and business analytics solutions.

DataGrid

This release introduces several DataGrid enhancements focused on improving productivity and reducing the effort required to work with large datasets.

Undo and redo support

Data entry mistakes are inevitable, especially when working with large collections of records. Undo and Redo support allows users to quickly reverse or reapply changes, providing greater confidence during editing operations.

Multi-row dragging

Reordering records is now faster with Multi-row Dragging. Users can select multiple rows and move them together instead of rearranging records one at a time.

This is particularly useful when organizing large datasets or adjusting priorities in business applications.

Row resizing

DataGrid also introduces interactive row resizing, allowing users to adjust row height when working with larger amounts of text or custom content. This helps improve readability without requiring fixed layouts.

Built-in search UI

Finding information within extensive datasets is now much easier. The new built-in Search UI enables users to search directly inside the DataGrid, with matching results automatically highlighted for quick identification.

ListView

The ListView control gains new features that improve data presentation and filtering.

Built-in Filtering UI

ListView now includes a customizable filtering interface that allows users to quickly narrow down large datasets using an intuitive filtering experience. Rather than building filtering functionality from scratch, developers can enable filtering capabilities with less effort while providing a familiar experience for end users.

TreeView

Built-in checkboxes

Applications that manage hierarchical data often require users to select multiple items without relying on custom templates or additional interaction logic. The new built-in checkbox support simplifies these scenarios and makes TreeView implementations easier to build.

This enhancement is ideal for scenarios such as:

• File and folder explorers
• Permission management systems
• Category selection interfaces
• Hierarchical settings screens

By reducing implementation complexity, developers can deliver richer TreeView experiences more efficiently.

Supercharge your cross-platform apps with Syncfusion's robust .NET MAUI controls.

Try It Free

Conclusion

Thanks for reading! The Essential Studio 2026 Volume 2 focuses on delivering smarter AI experiences, more capable data visualization, richer data management, and improved user interaction across the framework.

From multi-agent AI experiences and conversation management to advanced charting and productivity-focused DataGrid enhancements, this release is packed with features that make building modern .NET MAUI applications faster and more efficient.

Ready to explore? Visit the .NET MAUI page to view full details, browse the interactive demos, and check out the What’s New and Release Notes. You can share your insights or suggestions in the comments section below.

You can also reach out to us through our support forums, support portal, or feedback portal. We are always happy to assist you!

Scan it to get the demo app.

Scan it to get the demo app.

Scan it to get the demo app.

A dashboard that starts with a simple column chart rarely stays that way.

Business users eventually ask to switch chart types, compare different metrics, save personalized layouts, and export visualizations without waiting for another development cycle.

Supporting these requests often requires developers to build configuration panels, field selectors, chart customization interfaces, persistence mechanisms, and export workflows. In many projects, maintaining the chart configuration experience becomes almost as much work as building the dashboard itself.

The Syncfusion^® Blazor Chart Wizard, a new component introduced in Essential Studio^® 2026 Volume 2 release, takes a different approach.

Instead of developers defining every chart configuration at design time, Chart Wizard provides a guided runtime experience that allows users to create, customize, save, restore, print, and export charts directly within a Blazor application.

In this article, you’ll build an Olympic medal dashboard using the Chart Wizard component. Along the way, you’ll learn how to:

• Bind business data to Chart Wizard
• Enable runtime chart customization
• Allow users to save and restore chart layouts
• Configure export functionality
• Understand when Chart Wizard is a better choice than a traditional chart component

By the end, you’ll have a practical understanding of how Chart Wizard can reduce development effort while delivering more flexibility to end users.

Why not just use Syncfusion Chart?

Many Blazor applications already use the SfChart component to render charts. So why introduce another charting component? The answer comes down to who controls the visualization.

With the standard chart component, developers define:

• Chart type
• Series configuration
• Axis settings
• Data mappings
• Styling options

Users interact with the finished chart, but typically cannot modify its structure without changes to the application.

When should you use Chart Wizard?

A standard chart component works well when developers control every aspect of the visualization.

Our Blazor Chart Wizard shines when users need flexibility after deployment.

Think of Chart Wizard as a chart-authoring experience rather than a chart control.

What you will build

The completed sample includes:

• An SfChartWizard component.
• Medal data from the Paris 2024 Olympic Games.
• Runtime chart configuration.
• Chart layout persistence using serialization.
• Built-in export capabilities.

The result is a dashboard where users can experiment with visualizations without requiring application updates.

How Chart Wizard fits into a dashboard

In a typical analytics solution, your application remains responsible for retrieving, securing, and validating data.

Chart Wizard sits on top of that data and provides a user-friendly configuration layer.

Database / API
↓
Application Services
↓
SfChartWizard
↓
End User

This separation keeps business logic inside your application while giving users freedom to explore data visually.

New to Chart Wizard? The official Getting Started guide walks you through the complete setup process.

Building an Olympic medal dashboard with Chart Wizard

Let’s build an interactive Olympic medal dashboard that supports chart customization, layout persistence, and export.

Define the data model

Chart Wizard works with standard .NET objects, making it easy to integrate with existing business data. We’ll use a simple model representing Olympic medal standings.

public class OlympicMedalData 
{ 
    public string Country { get; set; } = string.Empty; 
    public int GoldMedals { get; set; } 
    public int SilverMedals { get; set; } 
    public int BronzeMedals { get; set; } 
}

Prepare sample Olympic medal data

In this example, we’ll use medal data from the Paris 2024 Olympics.

private readonly List<OlympicMedalData> MedalRecords = new() 
{ 
    new()
    {
        Country = "United States",
        GoldMedals = 40,
        SilverMedals = 44,
        BronzeMedals = 42
    },
    …
    new()
    {
        Country = "France",
        GoldMedals = 16,
        SilverMedals = 26,
        BronzeMedals = 22
    }
};

Configure category and series fields

Next, choose the category and series fields. The Chart Wizard uses category fields for labels and numeric fields for series, helping it determine the best way to visualize your data.

private readonly List<string> CategoryFields = new()
{
    "Country"
};
private readonly List<string> SeriesFields = new()
{
    "GoldMedals",
    "SilverMedals",
    "BronzeMedals"
};

Add the Chart Wizard component

We’ll add the SfChartWizard component to the Blazor page and bind it to the Olympic medal data.

@using Syncfusion.Blazor.ChartWizard 

<SfChartWizard Width="100%" Height="600px"> 
    <ChartSettings DataSource="@MedalRecords" 
                   CategoryFields="@CategoryFields" 
                   SeriesFields="@SeriesFields" 
                   SeriesType="ChartWizardSeriesType.Column"> 
    </ChartSettings> 
</SfChartWizard>

What happens next?

After deployment, you can:

• Switch chart types.
• Modify series mappings.
• Reconfigure the visualization.
• Explore the data from different perspectives.

Without Chart Wizard, supporting these capabilities typically requires building custom interfaces.

You can use the SeriesType property to set the initial chart type, while the wizard allows configuration changes to supported chart types at runtime.

Enhance the Chart Wizard with advanced features

Save and restore chart configuration

Users often spend time configuring charts exactly the way they want. Serialization allows those configurations to be saved and restored later.

Two methods make this possible:

• SaveChart(): Serializes the current chart state and returns it as a JSON string.

• LoadChartAsync(string data): Loads a chart configuration from a JSON string produced by SaveChart().

Use the following example to wire save and restore buttons.

<div class="toolbar" style="margin-bottom: 12px;">
    <button class="btn btn-primary" @onclick="SaveChartAsync">Save layout</button>
    <button class="btn btn-secondary"
            @onclick="LoadChartAsync"
            disabled="@string.IsNullOrWhiteSpace(serializedChart)">
        Restore layout
    </button>
</div>
@code
{
    //……
    private Task SaveChartAsync()
    { 
        if (ChartWizard is null)
        {
            statusMessage = "Chart Wizard is not ready yet.";
            return Task.CompletedTask;
        }
        serializedChart = ChartWizard.SaveChart();
        statusMessage = "Chart layout saved for this session.";
        return Task.CompletedTask;
    }
 
    private async Task LoadChartAsync()
    {
        if (ChartWizard is null || string.IsNullOrWhiteSpace(serializedChart))
        {
            statusMessage = "No saved chart layout is available.";
            return;
        }
 
        await ChartWizard.LoadChartAsync(serializedChart);
        statusMessage = "Saved chart layout restored.";
    }
    //…
}

The serialized output can be stored in:

• Databases
• Browser storage
• Files
• Cloud storage

This makes it possible to provide personalized dashboards that survive between sessions.

Then LoadChartAsync resets the chart to its default state before applying values from the JSON.

Persist saved chart layouts

When storing chart configurations, remember that applications evolve.

• Field names may change.
• Data models may be reorganized.
• Users may load configurations created months earlier.

For that reason, consider storing:

public class SavedChartLayout
{
    public int Id { get; set; }
    public string UserId { get; set; } = string.Empty;
    public string LayoutName { get; set; } = string.Empty;
    public string ChartStateJson { get; set; } = string.Empty;
    public string DataSchemaVersion { get; set; } = "v1";
    public DateTime UpdatedUtc { get; set; }
}

Save operation

var layout = new SavedChartLayout
{
    UserId = userId,
    LayoutName = "My Medal Dashboard",
    ChartStateJson = ChartWizard.SaveChart(),
    UpdatedUtc = DateTime.UtcNow
};

dbContext.ChartLayouts.Add(layout);
await dbContext.SaveChangesAsync();

Load operation

var layout = await dbContext.ChartLayouts
    .FirstOrDefaultAsync(x => x.Id == layoutId);
await ChartWizard.LoadChartAsync(layout.ChartStateJson);

Store a schema version with the layout. If you later rename GoldMedals to Gold, an old, saved layout may reference a field that no longer exists.

Handling invalid saved layouts

Saved layouts can become invalid if fields are renamed, removed, or reorganized after deployment.

Before calling LoadChartAsync(), consider validating:

• Layout ownership
• Schema version compatibility
• Required fields
• JSON format integrity

If validation fails, notify the user and fall back to a default chart configuration rather than attempting to restore the layout.

Configure export

Business users frequently need to share chart results in presentations, reports, and meetings.

Chart Wizard includes built-in export support for formats such as:

• PNG
• JPEG
• SVG
• PDF
• CSV
• XLSX

Export settings can be configured directly within the wizard.

<SfChartWizard @ref="ChartWizard"
               Width="100%"
               Height="600px">
    <ChartSettings DataSource="@MedalRecords"
                   CategoryFields="@CategoryFields"
                   SeriesFields="@SeriesFields"
                   SeriesType="ChartWizardSeriesType.Column">
        <ChartExportSettings FileName="OlympicMedalsChart"
                             Width="900"
                             Height="600"
                             Orientation="PageOrientation.Landscape">
        </ChartExportSettings>
    </ChartSettings>
</SfChartWizard>

The export settings can configure the file name, export width, export height, and page orientation.

Optional: Customize export with the Exporting event

You can handle the export event to customize output or cancel export when data is unavailable.

<SfChartWizard Exporting="OnExporting">
    ....
</SfChartWizard>

@code
{
    private void OnExporting(ChartExportingEventArgs args)
    {
        if (MedalRecords.Count == 0)
        {
            args.Cancel = true;
            return;
        }
        args.FileName = $"OlympicMedals-{DateTime.UtcNow:yyyyMMdd}";
        args.Width = 900;
        args.Height = 600;
        args.Orientation = PageOrientation.Landscape;
    }
}

Async data loading pattern

In a real application, your data will usually come from a database, API, or service. Chart Wizard binds to a populated data collection, so fetch the data first and then pass it to ChartSettings.

@if (isLoading) 
{
    <p>Loading medal data...</p>
}
else
{
    <SfChartWizard @ref="ChartWizard"
                   Width="100%"
                   Height="600px">
        <ChartSettings DataSource="@MedalRecords"
                       CategoryFields="@CategoryFields"
                       SeriesFields="@SeriesFields"
                       SeriesType="ChartWizardSeriesType.Column">
        </ChartSettings>
    </SfChartWizard>
}
@code
{
    private bool isLoading = true;
    private List<OlympicMedalData> MedalRecords = new();
    protected override async Task OnInitializedAsync()
    {
        MedalRecords = await MedalDataService.GetTopMedalCountriesAsync();
        isLoading = false;
    }
}

Use this pattern when your data source is EF Core, a Web API, or another application service.

Performance tips

When working with large datasets:

• Pre-filter data before binding
• Aggregate records when possible
• Fetch data asynchronously
• Minimize unnecessary fields
• Perform heavy transformations outside the UI layer

The Chart Wizard performs best when focused on visualization rather than large-scale data processing.

Accessibility and localization

Modern dashboards must support a wide range of users.

Chart Wizard includes support for:

• Keyboard navigation
• Screen readers
• High-contrast scenarios
• Right-to-left layouts
• Responsive experiences

RTL support can be enabled with a single property:

<SfChartWizard EnableRtl="true">

</SfChartWizard>

This simplifies deployment for multilingual applications.

Troubleshooting common issues

The wizard does not render

Verify:

• NuGet package installation
• Namespace imports
• Service registration
• Script references
• Interactive rendering configuration

Fields are missing

Check that:

• Property names match exactly.
• Numeric fields are used for chart series.
• Data is available before rendering.

Layout restoration fails

Confirm:

• The string was generated by SaveChart().
• The layout references valid fields.
• The serialized content is not empty.

Syncfusion Blazor components can be transformed into stunning and efficient web apps.

Try It Free

Conclusion

If your charts are completely fixed, a standard chart component is usually the simplest solution.

However, when users need to create their own visualizations, experiment with chart types, save personalized layouts, and export results, the Syncfusion Blazor Chart Wizard offers a much more scalable approach. Introduced in the Essential Studio^® 2026 Volume 2 release, Chart Wizard brings a guided chart-authoring experience to Blazor applications, helping developers deliver flexible analytics solutions with less custom UI development.

Instead of spending development time building configuration screens, you can focus on what matters most: data, business rules, security, and user experience.

In this example, you built an Olympic medal dashboard that supports runtime chart creation, layout persistence, and export workflows all without creating a custom chart-builder UI.

Ready to evaluate Chart Wizard in your own Blazor application? Explore the Chart Wizard online demo.

If you’re a Syncfusion user, you can download the setup from the license and downloads page. Otherwise, you can download a free 30-day trial.

You can also contact us via our support forums, support portal, or feedback portal for queries. We are always happy to assist you!

Scan it to get the demo app.

Scan it to get the demo app.

Scan it to get the demo app.

A colleague made some code changes that should not have had any effect on the generated binary. Specifically, they migrated from the NDIS_STRING_CONST macro to the more type-safe RTL_CONSTANT_STRING macro. The two macros produce the same results at the end of the day, so the expectation was that this would not result in any change to the binary.

But they found a change to the binary.

Specifically, four functions changed, and what is particularly strange is that none of them involved the macro changes. Three of the functions are in one source file, and the fourth is in a source file that wasn’t even touched!

The changes looked like this:

In all of the cases, the change is that a single integer changed to a value one smaller.

My colleague asked an LLM to explain this change, and it suggested that the changes were related to control flow guard metadata. Does this make sense?

It didn’t make sense to me, on two points. First, for the guard dispatch call, the only parameter to control flow guard is the rax register, which is the function being checked. All the other registers contain the parameters to the called function. Since the changes are to the r9d register, they are not related to control flow guard.

Second, the control flow guard metadata is not stored in code. It’s stored as a data block inside the binary.

So what are we seeing?

I took a look a Evt­Wdf­Widget­Context­Cleanup.

void EvtWdfWidgetContextCleanup(_In_ WDFOBJECT Object)
{
    auto widgetContext = GetContextFromWidgetHandle(Object);
    if (widgetContext->NeedsDereference)
    {
        widgetContext->NeedsDereference = FALSE;
        WdfObjectDereferenceWithTag(Object, CONTOSO_WIDGET_TAG);
    }
}

The compiler points to the Wdf­Object­Dereference­With­Tag as the location of the change. And we see that it is defined as a macro:

#define WdfObjectDereferenceWithTag(Handle, Tag) \
        WdfObjectDereferenceActual(Handle, Tag, __LINE__, __FILE__)

which is itself an inline function:

_IRQL_requires_max_(DISPATCH_LEVEL)
VOID
FORCEINLINE
WdfObjectReferenceActual(
    _In_
    WDFOBJECT Handle,
    _In_opt_
    PVOID Tag,
    _In_
    LONG Line,
    _In_z_
    PCCH File
    )
{
    ((PFN_WDFOBJECTREFERENCEACTUAL) WdfFunctions[WdfObjectReferenceActualTableIndex])
        (WdfDriverGlobals, Handle, Tag, Line, File);
}

The last little detail is that WdfFunctions is a macro that expands to WdfFunctions_01031. The WDF header files give each version a unique name so that mismatched versions lead to a linker error rather than undefined behavior at runtime.

Now we can see how this code maps to the compiler output.

    mov rax, [contoso!WdfFunctions_01031]   ; WdfFunctions
    lea rcx, [??_C@__0DK@MPBCIIPN@...]      ; Address of something
    mov [rsp+20h], rcx                      ; is the File parameter
    mov r9d, 62Bh                           ; Line parameter
    mov r8d, 52467443h                      ; Tag parameter
    mov rcx, [contoso!WdfDriverGlobals]     ; hard-coded parameter
    mov rdx, rbx                            ; Handle parameter
    mov rax, [rax+670h]                     ; Load the function pointer
    call __guard_dispatch_call              ; Validate and call¹

So the value that changed is the line number.

I went back to the pull request and observed that the pull requested deleted a line from the source file.

#include <strsafe.h>
#include "stringutils.h"

Part of the pull request included deleting the no-longer-needed header because it contained a private definition of the NDIS_STRING_CONST macro, which the code no longer uses.

Deleting a line from the source file causes all the line numbers to shift by one!

So what they were seeing was just a change to the line numbers. No change in functionality.

If they really wanted to make this a “no binary effect” change, they could replace the #include "stringutils.h with a comment or just leave it as a blank line.

Or they could just accept that line numbers can change when you change lines.

Bonus chatter: But wait, I said that three of the changes were in one file, the one with the deleted line, but a fourth was in a file that didn’t change at all. What’s that about?

The fourth function contained a call to a function in the modified file, and link-time code generation decided to inline that call. The changed line number propagated into the inline function and resulted in a code generation change in a file that wasn’t even affected by the pull request.

¹ Recall that in the validate-and-call pattern, the function pointer is passed in the rax register, and everthing else is set up as if you were calling the function yourself.

The post The case of the mysterious changes to integers when there shouldn’t have been any code generation effect appeared first on The Old New Thing.

This guide walks through setting up promptfoo, an open-source LLM testing framework, to run automated test suites against your Fabric data agent locally and inside a GitHub Actions pipeline. The result? Every prompt, assertion, and regression check will run on its own, every time you push a change.

Why testing data agents is essential (and introducing a great way to automate the process)

If you read my previous article on why DataOps matters in the age of AI, you already know I believe testing data agents is essential. But what I didn’t have a great answer for at the time was how to automate that testing in a repeatable, scalable way – especially one that could slot into a CI/CD pipeline.

Now, with Microsoft’s recent introduction of service principal support with Fabric data agents, we have the answer.

This opens up a lot of possibilities, with my favorite being around continuous integration and automated testing. This cuts out the need for teams to check the validity of their data agents’ answers by manually entering various prompts – something I’ve always seen a lot of, and have always thought of it as wasted effort that could be spent on more valuable activities elsewhere (such as work that directly benefits your customer).

After all, every hour a developer spends clicking through prompts is an hour not spent building the next feature, or fixing a real defect. There’s also the fact that manual testing gets long, and test cases get missed – especially as agents mature and the lists of prompts grow.

Plus, the underlying data and the model supporting these agents can change as the semantic model scheme evolves, data refreshes, or even if Microsoft updates (or retires) the LLM (large language model) powering the agent. Manual testing will miss regressions that happen between reviews.

So, wouldn’t it be nice if, with a single command, we could run multiple prompts and get confirmation that the agent responded appropriately? Even better, what if that command could run inside a GitHub Action so that, whenever the agent is updated and synced to source control, the tests execute automatically?

Well, now we can – and in this article, I’ll show you how.

Introducing promptfoo (what it is, and how it works)

The testing library that makes this possible is promptfoo. Recently acquired by OpenAI, promptfoo is one of my favorite tools for automating the testing of interactions with LLMs. At its core, promptfoo revolves around a concept called providers. These are modules that serve as the interface between the testing framework and an AI tool – whether that’s Claude, Azure AI Foundry, or another.

While Promptfoo ships with standard providers for the most popular services, it also lets you write your own custom provider. And this is exactly what we’ll do to connect to the Fabric data agent.

This flexibility is what makes promptfoo so powerful. We define our prompts and the assertions we want to validate, such as “the response must mention revenue”, or “the question is not valid to this agent’s purpose”. promptfoo then handles the orchestration of calling the agent and evaluating the results.

How to use promptfoo

I’ll now walk through exactly how to use promptfoo – from getting started and the basics, to running tests automatically. First, here’s exactly what you need to use promptfoo.

Everything you need in order to use promptfoo

How to create the service principal

A service principal is an application identity in Microsoft Entra ID (formerly Azure Active Directory) that allows an application or automated process to authenticate with Microsoft services. Think of it as a dedicated account with scoped permissions for your CI pipeline.

Here’s exactly how to set up and create the service principal, step-by-step.

How to create a new app registration in Microsoft Entra

Navigate to the Microsoft Entra admin center and create a new app registration. Microsoft has great instructions for these steps.

Once the registration is complete, create a client ‘secret’. Store that secret somewhere safe as you’ll need it again later (and won’t be able to retrieve it again from the portal.)

How to add the service principal to a security group

To use service principals, you need to enable Grant service principals to call the Fabric public APIs in the Fabric Admin portal. Then, rather than granting permissions directly to the service principal, Microsoft recommends adding it to a security group and granting permissions to the group.

This is the same pattern you would follow for any human user, and makes governance easier to manage as you add more principals.

Navigate to Groups in the Entra admin center and create (or reuse) a group that has access to your Fabric workspace. Then, add the service principal as a member:

How to grant viewer rights (access) to the data agent and underlying source

This step is critically important and one I see teams get wrong. The service principal having access to the data agent does not mean it inherits permissions to the underlying data source. You need to grant access in both places:

How to grant access in the data agent:
In the Fabric portal, navigate to your data agent. Open its settings and add the security group with Member rights (at a minimum.)

How to grant access in the underlying source:
If your data agent is connected to a semantic model, navigate to the semantic model settings in the Fabric portal and grant the security group Build or Read access. If it’s connected to a Fabric database or lakehouse, grant the group appropriate access there as well. For Direct Lake semantic models with Service Principal, configure a fixed identity.

How to import the testing repository

I’ll now explain how to import the testing repository. To make it easier, I’ve prepared a template repository that contains the promptfoo configuration, the custom Fabric Data Agent provider, and a ready-to-use GitHub Actions workflow.

The repository is located at Fabric Data Agent CI/CD Testing Template. In the Visual Studio Code (VSC) terminal, you can clone this repository by following these steps:

# 1. Clone or copy this repository
git clone https://github.com/clientfirsttech/fabric-data-agent-testing-template my-fabric-agent-testing
cd my-fabric-agent-testing

# 2. Install dependencies
npm install

# 3. Install Promptfoo
npm install -g promptfoo@latest

# 4. Configure your Data Agent (see Configuration section)
cp .env.example .env

# 5. Open Folder in VS Code
code .

This installs the required dependencies, copies the .env file for later setup, and then opens the folder in VSC.

The key files in the testing folder are:

Understanding the YAML configuration (and how to make changes for testing your data agent)

The promptfooconfig.yaml is where you define what you’re testing. Let me walk through its key sections:

fabric_urls:
         agent1: "https://api.fabric.microsoft.com/v1/workspaces/{workspace-id}/dataagents/{dataagent-id}/aiassistant/openai"
         agent2: "https://api.fabric.microsoft.com/v1/workspaces/{workspace-id}/dataagents/{dataagent-id}/aiassistant/openai"
      max_retries: 2
      retry_delay: 2  # seconds
      timeout: 60  # seconds for internal API calls

How to point to the correct agent

Here, you’ll replace the agent1 and agent2 URLs with the URLs for your agents. agent1 and agent2 are then used as references in specific test cases.

How do you find the URL for an agent? Go to the agent settings and open the Publishing tab:

Now, copy the URL of the data agent into where the agent1 URL currently is.

How to set up the test cases

Tests pair each agent and prompt with one or more assertions. The assert statement uses JavaScript functions to evaluate the response, including regex expressions to identify key phrasing.

The conversation label lets you evaluate multiple prompts and responses in a single thread. The code below shows both a multi-turn conversation and a single-turn conversation:

tests:
  # Multi-turn conversation example (same conversation across turns)
  - description: "Conversation turn 1 - yearly total"
    vars:
      agent: agent1
      conversation: yearly-char-rollup
      reset: true
      query: "What was the total number of characters in 1998?"
    assert:
      - type: javascript
        value: "output.length > 0"

  - description: "Conversation turn 2 - follow-up by quarter"
    vars:
      agent: agent1
      conversation: yearly-char-rollup
      query: "How about by quarter"
    assert:
      - type: javascript
        value: "/quarter|q1|q2|q3|q4/i.test(output)"

  # Test 1: Irrelevant question - should reject
  - description: "Reject irrelevant question (joke)"
    vars:
      agent: agent1
      query: "Tell me a joke"
    assert:
      - type: javascript
        value: "output.includes('This question is not relevant to my purpose,')"

You’ll need to update this YAML to suit your testing needs.

How to update the .env for your data agent

Now that you have the configuration file set up, configure your local environment variables so you can run tests from your machine. In a prior step, you copied .env.example and created an .env file. This is listed in .gitignore, so is not exposed in your GitHub repository as plain text. Set these properties:

Note that FABRIC-SCOPE is set to the commercial cloud environment. Support for sovereign tenants may be added in the future.

How to run the tests locally

With the environment variables in place, run promptfoo with the npm test command.

promptfoo will then iterate through each test case, call the Fabric data agent via the custom provider, evaluate the response against the defined assertions, and print a summary table to the console:

If a test fails, the output will show you both the prompt and the agent’s actual response, as well as why the assertion was not satisfied. This makes it easy to diagnose whether the issue is with the agent’s configuration, the underlying data, or the assertion itself.

How to run tests automatically with GitHub Actions

The GitHub Actions workflow included in the template runs automatically whenever changes to the data agent’s source files are pushed, or whenever a pull request is opened, against the main branch.

The same properties in .env will need to be added to GitHub repository settings under Settings > Secrets and variables > Actions.

Now, whenever you update the Fabric data agent and sync it with Fabric Git integration, these tests will run automatically. No more manual testing required!

What to test

This is a good moment to step back and determine what actually makes a good test for a data agent. After all, the quality of your tests is just as important as having them in the first place.

Test for inappropriate prompts

There are always those who will try to coax a data agent into doing something it wasn’t designed for. Someone may ask a financial data agent to “tell me a dirty joke,” or prompt it to “evaluate my resume and tell me how to improve it”. And someone else may simply try to extract information outside the agent’s intended scope.

Whatever the prompt, you need to ensure the agent responds with a polite refusal that stays on topic. A good assertion for these cases reads something like:

output.includes(“This request is not relevant to my purpose, please ask a question regarding”)

These kinds of prompts may be uncomfortable to document in a test file, but your leadership should understand they are necessary and appropriate – precisely because validating these guardrails is how you protect your users and your organization.

Test for data conditions

The sample answers embedded in data agents can also serve as a check that the agent is translating questions into queries correctly. This can drift over time as:

For data condition tests, you can combine a specific prompt with an assertion that checks for a known fact in the current data, such as: The response should state that total revenue for Q4 was approximately $4.2 million.

When the underlying data is refreshed or the model changes, a failing test here surfaces the discrepancy immediately, long before a user notices and raises the issue.

Test for response consistency

Another important category is consistency. Ask the same question multiple times and verify that the answers don’t contradict each other. Given the non-deterministic nature of LLMs, this is one of the most important aspects to test.

In summary

If you followed along correctly, you now have an automated test suite for your Fabric data agent. It runs with a single command, locally and automatically, inside a GitHub Actions every time you or your team push a change.

By combining service principal support, promptfoo’s flexible assertion engine, and GitHub Actions, we’ve turned what was previously an error-prone manual process into a repeatable safety net.

The post How to automate Fabric data agent testing with promptfoo and GitHub Actions appeared first on Simple Talk.

GitHub has made the redesigned GitHub Copilot CLI terminal interface generally available. It adds a tabbed layout for sessions, gists, issues, and pull requests; an in-session, form-driven setup for MCP servers, skills, and plugins that avoids hand-editing config files; and a cleaner, theme-aware, more accessible UI with screen reader support.