Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
157202 stories
·
33 followers

Automating cross-repo documentation with GitHub Agentic Workflows

1 Share

“Where are the docs?” It’s a question nobody on a product team enjoys answering. The honest reply is usually some variant of “behind.” A writer is staring at a closed pull request, trying to reverse-engineer what changed. The pull request’s author has already moved on. By the time the doc actually publishes, the feature has shipped, sometimes more than once.

That used to be us on the Aspire team (we’re a small team of 10 building dev tools for distributed apps). A few months back, we were trying to figure out how to safely bring AI into automations we already trusted. That’s when we discovered GitHub Agentic Workflows. I started bolting prototypes into microsoft/aspire.

Here’s what that bought us, in numbers pulled straight out of GitHub: for Aspire 13.3 and 13.4, 82 feature-docs pull requests merged at a median of 44.8 hours after the product pull request, every one of them reviewed by the engineer who shipped the feature. No new headcount. No process retraining. Just a different way of asking “who writes this?”

🔒 The constraint: cross-repo automation is the hard part

Our product lives in microsoft/aspire and our docs site lives in microsoft/aspire.dev—different repo, deploy target, and review chain. Most teams figure out same-repo automation pretty quickly; cross-repo automation is where things get sharp. Broad repo-scoped tokens belong in a museum, and any responsible security posture (ours included) restricts them accordingly. That’s a good thing. It’s also a real bottleneck if the place where you write the docs isn’t the place where you write the code.

The default workflow for years was:

  1. Engineer ships a feature in microsoft/aspire.
  2. Docs writer notices weeks later.
  3. Docs writer opens the pull request, reads the diff, and pings the engineer to clarify what changed.
  4. Engineer is on the next feature, vaguely remembers, replies with half the picture.
  5. Docs draft ships, sometimes against a release that’s already out.

This is the reverse-engineering tax. We needed automation that crossed repos without handing an agent a write-everywhere token. GitHub Agentic Workflows turned out to be the answer.

🤖 Why GitHub Agentic Workflows

GitHub Agentic Workflows is a project from the GitHub Next team that I keep describing to people as “GitHub Actions, but with a model as the work-item processor and guard rails that satisfy security review.” That’s reductive, but it’s close.

The shape of it:

  • You author a workflow as a single markdown file (.github/workflows/my-thing.md). YAML-style frontmatter on top, an English-language prompt underneath.
  • You run GitHub Agentic Workflows compile, and it generates a sibling .lock.yml (a normal GitHub Actions workflow) that you commit alongside.
  • At runtime, the workflow runs an agent against your prompt with a constrained toolset.
  • Critically, the agent doesn’t write to GitHub directly. It emits intent (a JSON blob describing the pull requests, issues, and comments it wants to create), and a separate, narrowly scoped job (the safe-outputs handler) materializes that intent against a per-workflow GitHub app.

That last bullet is the unlock. The agent gets read access and a prompt. Writes go through a tiny verifiable pipeline with explicit allow-lists. Security review nods. We ship.

💚 A small aside: kindred stacks

I love when the tools you’re using to build are built with the same tools you’re using to build with. The GitHub Agentic Workflows docs are built with Astro and Starlight. So is aspire.dev—Astro with Starlight, dressed up with the wider Starlight plugin ecosystem (astro-mermaid, starlight-llms-txt, starlight-sidebar-topics, starlight-image-zoom, the gorgeous @catppuccin/starlight theme, and more. Shout-out to Chris Swithinbank and the Starlight maintainers, the entire ecosystem feels designed by people who genuinely care).

There’s a real kinship there. The tool we use to automate docs and the docs site we automate into share the same foundation. Convenient, because the Mermaid sequence diagram in the next section renders the exact same way in both worlds.

The end-to-end pipeline

Here’s the flow we landed on. The protagonist is a workflow called pr-docs-check.md living in microsoft/aspire.

Sequence diagram showing an automated docs workflow: merging a feature pull request in microsoft/aspire triggers a GitHub Actions check that has an agent draft the documentation, open a draft pull request in microsoft/aspire.dev, and request SME review—so docs ship with the feature.

A run starts on pull_request: closed against main or release/*, gated by merged == true. From there, the workflow first runs a deterministic target branch resolver in plain bash before the agent ever wakes up:

  1. Pull request milestone title (e.g. 13.4 → release/13.4 on aspire.dev).
  2. Linked-issue milestone title (parse Fixes/Closes/Resolves #N from the body, fetch each issue, take the first non-empty milestone).
  3. Pull request base ref, if it matches release/X.Y[.Z].
  4. Fall back to main.

This is the linchpin. Milestones in the product repo map cleanly to release branches in the docs repo. When the agent finally runs, it knows exactly where the docs should land without any creative writing about target branches or guessing.

The agent reads the diff, scans linked issues, and decides: does this need docs? If yes, it drafts the actual content in a checked-out microsoft/aspire.dev workspace, following our existing doc-writer skill (voice, MDX conventions, Starlight components). It then emits a create_pull_request safe-output and hands off.

The safe-outputs handler takes over:

  • Title prefix: [docs]
  • Label: docs-from-code
  • draft: true (we never auto-merge)
  • Base branch: agent-supplied, restricted to main or release/*
  • Target repo: microsoft/aspire.dev
  • Reviewer: the SME identified from the source pull request’s reviews—i.e., whoever the product team trusted to approve the feature, now gets asked to approve the doc for that feature.

A companion job posts a marker comment back on the source pull request with the docs pull request link and minimizes any older pr-docs-check comments on re-run. The engineer who just hit Merge gets a notification within a few minutes: “Here’s the docs draft. Look it over?”

🔐 The safe-outputs contract

The whole security story comes down to a small, boring stretch of frontmatter:

tools: 
  github: 
    toolsets: [repos, issues, pull_requests] 
    min-integrity: approved          # only run pinned, integrity-checked actions 
    allowed-repos: 
      - microsoft/* 
    github-app: 
      app-id: ${{ secrets.ASPIRE_BOT_APP_ID }} 
      private-key: ${{ secrets.ASPIRE_BOT_PRIVATE_KEY }} 
      owner: "microsoft" 
      repositories: ["aspire.dev", "aspire"] 

safe-outputs: 
  create-pull-request: 
    title-prefix: "[docs] " 
    labels: [docs-from-code] 
    draft: true                      # human-in-the-loop, always 
    base-branch: main 
    allowed-base-branches: [main, release/*] 
    target-repo: "microsoft/aspire.dev" 
    protected-files: blocked         # AGENTS.md, manifests, security config: hands off 
    fallback-as-issue: true 

That’s the deal in plain text. The agent gets a GitHub App token whose installation is scoped to exactly two repositories—the product repo and the docs repo—and nothing else in the org is reachable. It can only land pull requests against main or release/*. AGENTS.md and dependency manifests are off-limits by policy. If the pull request creation fails (network blip, conflict, anything), the framework falls back to filing an issue, so nothing is silently dropped.

This is the part security review actually liked. The agent’s reasoning is fuzzy. The action surface is not.

📊 By the numbers

Here are the stats from a rolling 30-day window (May 3 – June 2, 2026) spanning the back end of the Aspire 13.3 release and the run-up to 13.4:

Metric Value 
Product pull requests merged in microsoft/aspire 396 (338 main / 50 release/13.3 / 8 release/13.2) 
pr-docs-check workflow runs 396 
Draft docs pull requests created on microsoft/aspire.dev 82 
  – Merged 82 (100%) 
  – Closed without merge 
  – Still open 
Docs pull requests target branches 52 → release/13.3, 27 → release/13.4, 3 → main 
Median time-to-merge (docs) 44.8 hours 
Merged within 24 h / 7 days 38% / 96% 

Note: Numbers captured at the time of writing; the workflows keep running, so the totals only go up. 

A few of those numbers deserve a second look:

  • 396 runs → 82 pull requests is not a defect. The workflow runs on every merged pull request; most of them are internal refactors, test fixes, or dependency bumps with no user-facing surface. The agent saying “no docs needed” 300+ times is a feature.
  • 100% merge rate says the agent’s docs picks are right. The tighter prompt we shipped after the v1 false-positive phase is paying off.

✅ What worked, what didnt

What worked

  • Milestone → release-branch mapping. This was the single highest-leverage choice we made. Engineers already set milestones on pull requests and issues; we got accurate target-branch routing for free.
  • Draft-only, SME-as-reviewer. The agent never merges. The engineer who shipped the feature is the one who confirms the docs are right. We’ve stopped reverse-engineering features at the doc layer. The engineer just tells the docs draft what to say, in the place where they already are.
  • Scoped GitHub app per workflow. Each workflow gets its own app token with explicit repo and permission scopes. Security review approved. We approved too; the first time we needed to rotate keys.
  • protected-files: blocked. The agent cannot touch AGENTS.md, package manifests, or repo security config. Period.

What didn’t (at first)

  • ❌ The agent’s “is this docs-worthy?” gate was too generous in the first version. It drafted pull requests for changes that were genuinely internal, such as a CI tweak or a logging refactor. The result: 9 closures of 69 pull requests (≈13%), so we tightened the prompt’s user-facing-change definition and added explicit negative examples (CI, internal helpers, tests-only). Now, the rate is trending down.
  • ❌ Cross-repo pull request creation needed a mirrored checkout pattern that wasn’t obvious from the docs. The agent works in one repo; safe-outputs needs to find the target repo to push a branch. We solved it by checking out microsoft/aspire.dev twice—once as the current workspace, once under _repos/aspire.dev—so the safe-outputs handler can rediscover it deterministically.
  • ❌ Big diffs blow prompt budgets. We pre-extract pull request metadata (linked issues, milestone, base ref) in pre-agent-steps bash, so the agent gets a small, structured summary instead of a giant payload. This is GitHub Agentic Workflow’s designed-in pattern, and it works.

Wrapping up

The changes we made shifted our thinking. A feature wasn’t considered done until the docs were. Docs no longer trail along behind it like a tin can on a string. The engineer’s review is the gate; the bot does the typing.

Critically, this doesn’t replace docs writers; it un-burdens them. Our writers used to spend most of their time reverse-engineering features. Now they spend their time on the things only a human can do well: narrative pages, sample programs, conceptual walkthroughs, the parts of the docs that don’t fall out of a diff. The bot handles the mechanical “this new option was added; here’s the reference page update” work that was never enjoyable for anyone.

Huge thanks to the GitHub Next team for GitHub Agentic Workflows (and for making the safe-outputs primitive a first-class part of the design), and to Chris Swithinbank and the Starlight maintainers for the docs platform we automate into. A genuine thank-you, too, to the security folks whose guardrails forced us to design this the right way the first time. The boring secret of good automation is that strong security constraints make the system more trustworthy and more correct.

If you build a product in one repo and ship docs in another—and especially if you have to do it inside any nontrivial security boundary—GitHub Agentic Workflows is worth a serious look. Start with one workflow, such as pr-docs-check, and watch what happens to your median time-to-docs.

🔗 The other workflows

pr-docs-check is the one I wrote this post about, but it’s not running alone. If you’re curious about the rest, the source is public:

  • milestone-changelog.md: runs every two hours, picks up newly merged pull requests in the active milestone, and maintains a 13.x-Change-log wiki page (new features, improvements, notable bug fixes) with a companion editorial-feedback issue. 346 runs.
  • release-update-support-mdx.md: on a stable Aspire release, drafts a [support] pull request on aspire.dev that updates the support policy page (promotes the new version, demotes the previous one, refreshes the “Last updated” badge).
  • update-integration-data.md: lives in the docs repo; runs pnpm update:all daily, refreshes NuGet metadata + GitHub stats + sample data, and opens a chore: Update integration data PR with supersede-and-close logic for stale runs. 27 runs, eight merged pull requests.
  • repo-pulse.md: a rolling three-day repo dashboard pinned to a single issue and updated in place: recent merges, pull requests awaiting review, new issues, discussion activity. One issue, always fresh.

Happy automating, friends! 🤖🚀

The post Automating cross-repo documentation with GitHub Agentic Workflows appeared first on The GitHub Blog.

Read the whole story
alvinashcraft
just a second ago
reply
Pennsylvania, USA
Share this story
Delete

How to Build an MCP Server with FastMCP for Your Local AI Agent

1 Share

In this tutorial, I'll show you how to build an MCP server with FastMCP, connect your local AI agent to use tools from the local MCP server that you built, and add support for remote MCP servers. We'll wire the whole thing together with LangChain v1, Ollama, Qwen, and Python.

Model Context Protocol (MCP) is the common language between AI agents and tools. It's the standard way to expose tools to AI agents.

More companies are starting to expose MCP servers alongside their existing APIs, because MCP gives LLMs and AI agents a standard way to discover and use those capabilities directly.

Table of Contents

Background

A lot of simple local AI agents define their tools directly inside the same Python script as the agent. These are specific to the agent and every new agent has to re-implement the same tools from scratch.

MCP improves this by giving tools a standard interface that any MCP-compatible client can use. Write the tool once as an MCP server, and any compatible client can reuse it. And because MCP is a network protocol, those tools don't even have to run on your machine. Someone else can host an MCP server, and your agent can use its tools the same way it uses your local ones.

To follow this tutorial, you'll need Ollama installed on your machine. The tutorial works on macOS, Windows, and Linux. I'm using a MacBook Pro with 32 GB of RAM, but you can run this on a lower-memory machine by choosing a smaller Qwen model from Ollama.

What is MCP?

MCP (Model Context Protocol) is an open protocol that exposes tools, resources, and prompts to LLM clients.

Just as REST standardized many web APIs, MCP is the standardizing protocol for AI tools. Instead of every framework inventing its own tool interface, MCP defines a shared one, and anything that understands the protocol can use tools exposed by any MCP-compatible server.

The below image from modelcontextprotocol.io captures the idea well.

image from modelcontextprotocol.io that shows how MCP protocol connects AI applications to data sources and tools

An MCP server is a small program that exposes a list of tools. An MCP client is anything that connects to that server (for example, an AI agent) and lets an LLM call those tools.

MCP servers are commonly exposed over transports like:

  • stdio: the server runs as a subprocess of the client, communicating over stdin/stdout. Best for local tools that only your agent needs.

  • http: the server runs as an HTTP service and clients connect over the network. Best for shared or remote tools.

The protocol standardizes how tools are exposed so different AI agents and clients can use them consistently.

What is FastMCP?

FastMCP is a Python library that makes writing an MCP server feel like writing a FastAPI app. You decorate functions with @mcp.tool, and FastMCP handles the protocol details: JSON-RPC messages, tool schema generation from your type hints and docstrings, and the transport layer.

On the LangChain side, langchain-mcp-adapters is a library that connects to one or more MCP servers and loads their tools into a format LangChain v1's create_agent can use directly. The agent code doesn't know if a tool lives in a subprocess on your machine or on a remote server. It just sees a list of tools with names and descriptions.

Motivation and Architecture

The motivation behind this project is to create sharable tools and to reuse tools others have already built. I wanted to create tools like current_time and word_count and share them across every agent I build. I also wanted to use tools from public MCP servers for capabilities I don't want to write myself, like browsing GitHub repos.

Using a local LLM means my conversations never leave my machine. The only thing that touches the network is whatever the model decides to send to remote tools, and only when it decides to call them.

For this project, I'll use FastMCP to build a local MCP server with two tools, connect to DeepWiki's free public MCP server for GitHub repo lookups, use langchain-mcp-adapters to load both into a LangChain v1 agent, and Ollama to run the local Qwen model.

The flow has three processes.

  1. The local MCP server is a standalone Python script that exposes current_time and word_count. It runs as a subprocess of the agent, over stdio.

  2. The remote MCP server is DeepWiki's public service that exposes three tools (read_wiki_structure, read_wiki_contents, ask_question) for asking questions about any GitHub repo, over HTTP.

  3. The agent is the coordinating script that connects to both, merges their tools into a single list, and runs the interactive loop.

When the user asks a question, the model sees all tools from both servers as one list and picks whichever ones it needs.

Step 1: Install Ollama and Pull the Model

To get started, install the Ollama application for your platform.

We'll use Qwen as the chat model. Qwen has native tool-calling support, which is what makes it work well with MCP tools. I'm using qwen3.5:4b. If your machine has less RAM, you can use qwen3.5:0.8b.

ollama pull qwen3.5:4b

Step 2: Install Python Dependencies

python3 -m venv venv
source venv/bin/activate
pip install fastmcp langchain langchain-core langchain-ollama langchain-mcp-adapters

This tutorial requires langchain>=1.0.0.

Step 3: Build the Local MCP Server with FastMCP

The local MCP server exposes two small utility tools: current_time for checking the current date and time, and word_count for counting words in a piece of text. Any MCP client can use them, not just this agent.

FastMCP generates each tool's schema automatically from the type hints and docstrings, so the docstring wording matters. That's what the LLM sees when deciding whether to call each tool.

Save the code in your mcp_server.py file.

from datetime import datetime
from fastmcp import FastMCP

mcp = FastMCP("local-tools")


@mcp.tool
def current_time() -> str:
    """Return the current local date and time.
    Use this when the user asks what time or date it is.
    """
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")


@mcp.tool
def word_count(text: str) -> int:
    """Count the number of words in a piece of text.
    Use this when the user asks how long a piece of writing is
    or asks you to count the words in something they've shared.
    Returns the word count as an integer.
    """
    return len(text.split())


if __name__ == "__main__":
    # Run the MCP server over stdio.
    mcp.run()

Since this tools_server.py will be run in stdio mode as a subprocess, we don't need to start it separately. The agent will run it automatically.

Step 4: Agent Python Code

The agent code does three things. First, the configuration at the top defines the model, the system prompt, and the URL of the remote MCP server. The build_agent() function connects to both MCP servers, loads their tools into a single list, and creates a LangChain v1 agent. The main() function runs the interactive loop.

The [tool call] log line lets us see exactly which tool (local or remote) the agent picked on each turn.

Finally, await is used because build_agent(client) is asynchronous. It needs to wait for async MCP operations like client.get_tools() before it can return the finished agent. Without await, we would just get a coroutine object instead of the actual agent.

Save the code in your agent_with_mcp.py file:

import asyncio

from langchain.agents import create_agent
from langchain_ollama import ChatOllama
from langchain_mcp_adapters.client import MultiServerMCPClient

# Local Ollama model to use for the chat agent.
CHAT_MODEL = "qwen3.5:4b"

# Hosted remote MCP server we'll connect to over HTTP.
DEEPWIKI_MCP_URL = "https://mcp.deepwiki.com/mcp"

# System prompt that tells the model what tools it has and how to behave.
SYSTEM_PROMPT = (
    "You are a helpful assistant with access to tools for checking the current time, "
    "counting words, and looking up information about GitHub repositories. "
    "Use tools when the user's request needs information you don't already have. "
    "If a tool returns an error, tell the user plainly and do not retry with made-up arguments. "
    "If the question doesn't need a tool, just answer directly."
)


async def build_agent(client: MultiServerMCPClient):
    # Load tools from all connected MCP servers.
    # This is async because MCP communication happens over I/O.
    tools = await client.get_tools()
    print(f"Loaded {len(tools)} tools: {[t.name for t in tools]}")

    # Create the local Ollama chat model.
    model = ChatOllama(model=CHAT_MODEL, temperature=0)

    # Build a LangChain agent with the local model and all MCP tools.
    return create_agent(
        model=model,
        tools=tools,
        system_prompt=SYSTEM_PROMPT,
    )


async def main():
    # Create one MCP client that connects to two servers:
    #
    # 1. "tools" is a local MCP server started as a subprocess over stdio.LangChain will launch `python mcp_server.py` for us.
    # 2. "deepwiki" is a hosted MCP server we connect to over HTTP.
    client = MultiServerMCPClient({
        "tools": {
            "command": "python",
            "args": ["mcp_server.py"],
            "transport": "stdio",
        },
        "deepwiki": {
            "url": DEEPWIKI_MCP_URL,
            "transport": "streamable_http",
        },
    })

    # Build the agent after the MCP client is ready and tools are loaded.
    agent = await build_agent(client)

    print("\nReady! Ask the agent something.")
    print("Type 'exit' to quit.\n")

    while True:
        question = input("You: ").strip()
        if not question or question.lower() in {"exit", "quit"}:
            break

        # Send the user's message to the agent.
        # We use `ainvoke()` because the agent may call async MCP tools.
        result = await agent.ainvoke({
            "messages": [{"role": "user", "content": question}],
        })

        # Walk through the returned messages and print any tool calls
        # the agent made during this turn.
        for msg in result["messages"]:
            tool_calls = getattr(msg, "tool_calls", None)
            if tool_calls:
                for call in tool_calls:
                    print(f"[tool call] {call['name']}({call['args']})")

        # The final message in the list is the agent's final answer.
        print(f"\nAnswer: {result['messages'][-1].content}\n")


if __name__ == "__main__":
    # Run the async program.
    asyncio.run(main())

Step 5: Run the Agent

python agent_with_mcp.py

You don't need to start the local MCP server yourself. MultiServerMCPClient launches mcp_server.py as a subprocess over stdio, and also opens an HTTP connection to DeepWiki. If either server is unreachable, you'll see an error during startup rather than a silent fallback.

Once the agent is running, you can ask it questions in plain English. Before trusting the answers, watch the tool calls to make sure the agent picked the right tool with the right arguments. Local models are smaller than hosted frontier models and tend to hallucinate more. Spot-checking helps.

As a test run, I asked the agent a mix of questions:

$ python agent_with_tools.py

Starting MCP server 'local-tools' with transport 'stdio'                                                      transport.py:210
Loaded 5 tools: ['current_time', 'word_count', 'read_wiki_structure', 'read_wiki_contents', 'ask_question']

Ready! Ask the agent something.
Type 'exit' to quit.

You: what is the current time
[tool call] current_time({})

Answer: The current time is 2026-07-01 16:41:42

You: Give me one line summary of karpathy/nanochat 
[tool call] ask_question({'repoName': 'karpathy/nanochat', 'question': 'Give me a one-line summary of this repository'})

Answer: This repository, `karpathy/nanochat`, is a minimal, full-stack experimental system for training large language models (LLMs) from scratch, designed to be accessible and cost-effective, with a primary development focus on optimizing the "Time-to-GPT-2" benchmark.

You: what's the capital of France?

Answer: Paris

The agent behaved reasonably well for a 4B local model. It called current_time tool for the time question and reached out to DeepWiki's remote ask_question tool to answer a question about the nanochat repo. It also skipped tool calls entirely for the France question.

You can explore more MCP servers in the MCP server registry: https://github.com/modelcontextprotocol/servers

Conclusion

In this tutorial, we built an MCP server with FastMCP, connected to a free public remote MCP server, and wired both into a local AI agent using LangChain v1's create_agent and langchain-mcp-adapters.

From here, try adding your own tools to the local server, like a note reader or a wrapper around another local capability. Point the agent at other remote MCP servers. Or turn your local server into a remote one by switching its transport to HTTP and running it on a small server, so you can use it from any device you own or even publish it for others to use. Happy tinkering!

If you enjoyed this tutorial, you can find more of my writing on my blog (recent posts include system design paper series), my work on my personal website, and updates on LinkedIn.



Read the whole story
alvinashcraft
12 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Microsoft Testing Platform and .Net 10

1 Share

Microsoft Testing Platform and .Net 10

I created a new project using .Net 10 and it failed in my pipeline when running the tests. I wasn’t aware of the Microsft Testing Platform (MTP) and how it has changed in .Net 10.

What is MTP?

The Microsoft Testing Platform “is a lightweight and portable alternative to VSTest for running tests in all contexts, including continuous integration (CI) pipelines, CLI, Visual Studio Test Explorer, and VS Code Test Explorer. MTP is embedded directly in your test projects, and there’s no other app dependencies, such as vstest.console or dotnet test needed to run your tests.” It is open source and available on GitHub . Checkout the pillars in this link. It looks like it’s going in a great direction.

Read the whole story
alvinashcraft
40 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Using SpecKit and Specification-Driven Development to Tame AI Coding

1 Share

Using SpecKit and Specification-Driven Development to Tame AI Coding at NE Code 2026


✅ Presented at Nebraska.Code() 2026
📅 July 23, 2026
🎤 Session: Using SpecKit and Specification-Driven Development to Tame AI Coding


Introduction

AI is changing how we write software. According to DORA’s 2025 report, 95% of developers are now using AI in some capacity. Over the last year, we’ve been having a lot of conversations at work and I’ve been thinking a lot about how to use AI effectively in software development.

Read the whole story
alvinashcraft
51 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Beyond Prompts: Teaching AI How to Build with DevExpress

1 Share

AI Needs More Than Good Prompts

If you've paired Claude Code, GitHub Copilot or Cursor with DevExpress, you already know the moment I'm about to describe. It's the same one the team kept running into while building the DevExpress Office File API and Reporting Skills, and their evaluation notes are what this post is really about. The assistant starts strong. Clean C#, sensible class names, code that looks like it belongs in your solution. Then you read it a second time and something's off: the namespace is from a release two versions back, the API has been superseded, or the whole thing has quietly wandered off to a third-party library because that's what the model saw most often in training.

The interesting part wasn't that the AI got things wrong. Everyone expects that. What struck me about the team's findings was how convincing the wrong answers were. The ones that failed to compile cost them nothing; those get spotted immediately. The expensive ones looked completely reasonable right up until someone compared them with the current documentation. More often than not, they were yesterday's best practice dressed up as today's.

That single observation ended up shaping how the team designed the Skills.

As they worked through the results, the same theme kept coming back. When the generated code missed, the problem was almost never C#, and it certainly wasn't the model's ability to reason. It was product knowledge. The model didn't have the current namespaces, the right entry points or the small amount of framework-specific context it needed before it started typing. Hand it those things and the quality of the output jumped straight away.

We spend a lot of energy on prompt engineering, and good prompts genuinely help. But a prompt can only work with what the model already knows. No amount of clever wording fills a gap in product knowledge that isn't there to begin with. Every time, giving the assistant accurate context before it wrote anything beat trying to talk it back from a bad first answer.

That's the whole idea behind DevExpress AI Skills. They don't replace the model's reasoning, and they don't try to re-host the documentation. They hand the assistant a concise, product-specific starting point before it writes the first line of code, so it can spend its effort solving your actual problem instead of reconstructing an unfamiliar API from scraps of training data.

The easiest way to show you what I mean is to walk through the same prompts the team used during the review, with and without the Skill.


When Good Code Is Still Wrong

This showed up early in the team's evaluation. Completely made-up APIs were never the issue, because you catch those instantly. The answers that slowed them down were the believable ones: code that read like it came from someone who knew the framework reasonably well, but carried just enough stale or incorrect detail to send you chasing the wrong problem for half an hour.

That's the trap with specialised frameworks. They move constantly. Namespaces get reorganised, newer APIs replace older programming models, and products grow in directions that simply aren't in the model's training data yet. The assistant has no way of knowing a better approach exists unless something tells it, so it does the reasonable thing and fills the gap with the closest match it can find. Most of the time that match looks plausible enough to trust.

Across the Office File API and Reporting review, the team watched this happen again and again. Same prompt, same model, same task. The only thing they changed was what the assistant knew before it started. And the moment it started from accurate DevExpress guidance rather than memory, the code changed.

Here are three examples straight from that work.

Example 1: Generating a QR Code

Start with something ordinary.

Prompt

Generate a QR Code containing https://community.devexpress.com and save it as a PNG using DevExpress.

Without the Barcode Skill, the assistant reached for an older DevExpress API. The important thing to note, the Barcode functionality was NOT removed, simply improved and updated.

using DevExpress.BarCodes;

BarCode barCode = new BarCode();
barCode.Symbology = Symbology.QRCode;
barCode.CodeText = url;
barCode.Save("qr.png");

There's almost nothing to object to here. It reads cleanly, it follows normal C# conventions, and if you didn't already know the current Barcode API you'd have every reason to assume it was fine. That's exactly what makes this kind of error awkward. It doesn't look absurd. It looks sensible.

The trouble is the historical DevExpress.BarCodes namespace and an object model that no longer reflects the current API. The model hasn't invented nonsense; it has rebuilt a solution out of older fragments that showed up often enough in training to feel right.

With the Barcode Skill loaded, the same prompt lands on the current API.

using System.IO;
using DevExpress.Docs.Barcode;
using DevExpress.Drawing;

var qrOptions = new QRCodeOptions {
    ModuleSize = 4f,
    Dpi = 96
};

using var generator = new BarcodeGenerator(qrOptions);
using var output = new FileStream("qr.png", FileMode.Create, FileAccess.Write);
generator.Export("https://community.devexpress.com", output, DXImageFormat.Png);

Nothing changed except what the assistant knew going in. That matters more than it first appears, because the first answer becomes the foundation for everything after it. Start the conversation on an obsolete namespace and every follow-up (styling the barcode, switching the output format, dropping the image into another document) tends to stay on that same dead branch.

I'll be honest: these errors worry me more than a broken build ever has. A bad method name announces itself. Convincing code built on an old API doesn't, and it's easiest to trust precisely when you're moving fast and not looking closely. The Skill heads that off by putting the assistant on the right API before the conversation picks up speed.


Example 2: Exporting Large Excel Files

This one's trickier, because the "wrong" code isn't wrong in the usual sense. It compiles. It produces a perfectly valid workbook. The problem is that the assistant picked the wrong DevExpress product for the job.

Prompt

Export 100,000 rows to an Excel workbook efficiently using DevExpress.

Without the Excel Export Skill, most assistants reach for the Spreadsheet API.

using var workbook = new Workbook();

Worksheet sheet = workbook.Worksheets[0];

for (int i = 0; i < 100_000; i++)
{
    sheet.Cells[i, 0].Value = data[i];
}

workbook.SaveDocument("output.xlsx", DocumentFormat.Xlsx);

Functionally, this is a fair answer. It builds a workbook and writes data into it, and for most day-to-day work the Spreadsheet API is exactly what you want: it gives you a rich in-memory object model for creating, editing, formatting, calculating and analysing content.

But look at the prompt again. It says efficiently.

Building an Excel document and streaming out a large dataset are related jobs, not the same job. The Spreadsheet API holds the whole workbook in memory so everything stays editable for the life of the document. That's a strength when you're editing. At 100,000 rows it's mostly overhead, because once a row is written you're never going back to touch it, the formulas usually don't need recalculating mid-generation, and what you actually care about is memory that stays flat as the row count climbs. That's the exact problem the DevExpress Excel Export Library was built for.

With the Excel Export Skill in play, the assistant switches to the streaming API.

using DevExpress.Export.Xl;

IXlExporter exporter = XlExport.CreateExporter(XlDocumentFormat.Xlsx);

using var document = exporter.CreateDocument(stream);
using var sheet = document.CreateSheet();

foreach (var customer in customers)
{
    using var row = sheet.CreateRow();
    using var cell = row.CreateCell();
    cell.Value = customer.Name;
}

These two libraries aren't rivals. They solve different problems. One hands you a full workbook object model for manipulation; the other writes rows straight to the stream and keeps memory predictable no matter how big the export gets. On its own, the assistant has no real basis for preferring one over the other, because both technically satisfy the loose wording of the prompt.

That's the gap the Skill fills. It isn't fixing syntax; it's supplying the engineering judgement to match the API to the workload.


Example 3: Reporting Setup and Viewer Customisation

Reporting is a great stress test for AI-assisted development, because so much of it hinges on small platform-specific details. The code can look structurally perfect and still be missing the one registration call, callback or client-side API that makes the viewer actually work.

One evaluation prompt asked for the native DevExpress Report Viewer in a Blazor Server app. Without the Reporting Skill, the assistant registered services for the JavaScript-based viewer family instead of the native Blazor one.

builder.Services.AddDevExpressBlazorReporting();

Easy mistake to make. The method name sounds right, it slots neatly into the ASP.NET Core service registration pipeline, and nothing about it raises a flag. But native Blazor Reporting needs a different call.

builder.Services.AddDevExpressServerSideBlazorReportViewer();

This is the kind of thing that costs a disproportionate amount of pain. The app compiles, the page loads, and you're left staring at a blank viewer wondering what you did wrong. Nothing failed at the C# level. It failed at knowing which DevExpress Reporting viewer family the prompt was even talking about.

The same story plays out when you customise the viewer. Say you want to hide the Print button and limit exports to PDF. With no product guidance, assistants tend to invent APIs that feel plausible but don't exist.

previewModel.GetToolbarItemCommands().forEach(command => {
    if (command.id === "print") {
        command.visible = false;
    }
});

previewModel.GetExportFormatItems().forEach(item => {
    if (item.format !== "pdf") {
        item.visible = false;
    }
});

It reads like it belongs in a reporting viewer: sensible names, a clean split between toolbar and export customisation, the shape of a real client-side API. But the actual DevExpress callbacks use different entry points.

function customizeMenuActions(s, e) {
    var printAction = e.GetById(DevExpress.Reporting.Viewer.ActionId.PrintPage);
    if (printAction) {
        printAction.visible = false;
    }
}

function customizeExportOptions(s, e) {
    e.HideFormat(DevExpress.Reporting.Viewer.ExportFormatID.XLS);
    e.HideFormat(DevExpress.Reporting.Viewer.ExportFormatID.XLSX);
}

This is why specialised frameworks are hard for general-purpose assistants. The answer has the right shape and the wrong product knowledge, and in reporting, where so much of the setup is declarative or callback-driven, one wrong method name is the difference between a working viewer and a dead end.


When the Best Answer Is Another Question

Not every win comes from better code. Some of my favourite Skill behaviour shows up when the assistant decides not to write code yet.

AI assistants are heavily biased towards answering the prompt in front of them. When a requirement is underspecified, they'll usually make an assumption and press on. That keeps things moving, but it also produces software that satisfies the words of the prompt rather than what you actually meant.

Picture asking for "a chart using DevExpress." From a developer's point of view that request is wide open. Bar, line, pie, scatter? What's the data source? Who's the audience? Any experienced developer would ask a question or two before writing a line.

Most assistants won't. They'll pick whatever chart type seems most likely and start generating.

Several DevExpress Skills nudge the assistant the other way: when something essential is missing, ask first.

Developer

Create a chart using DevExpress.

AI with the Skill

Which chart type would you like to create? For example, Bar, Line, Pie, Area or Scatter?

That small shift makes the whole thing feel less like poking a code generator and more like working with a colleague. And it saves you the classic time-sink of unwinding an implementation built on an assumption nobody ever made.

You could bake this into your own prompts, of course, but then you're on the hook to remember it, repeat it across every assistant you use, and keep it in step with how your team works. Putting it in a Skill makes it part of the workflow instead of one more line you have to paste into every request.


Skills and the DevExpress MCP Server

One question comes up almost every time I talk about this:

If the DevExpress MCP Server already gives AI agents access to the documentation, why do we need Skills too?

Because they're solving different problems.

A Skill is the focused guidance the assistant gets before it starts: enough to recognise the right product area, follow the common implementation patterns, and sidestep the mistakes that come from leaning on training data alone. Skills are deliberately small, so they load fast and cover a lot of everyday work without sending the assistant off on several rounds of exploration.

The MCP Server plays a different role. It gives compatible agents live access to the current documentation, API details and examples. That's the shipping product, not whatever the model happened to memorise.

A developer prompt passes through a DevExpress Skill to the AI coding assistant, which exchanges requests and authoritative detail with the DevExpress MCP Server before producing generated code. A Skill points the assistant to the right product area up front; the MCP Server supplies authoritative detail once the task gets specific.

In practice they hand off to each other. The Skill gets the assistant to the right place; when the task needs more depth, the MCP Server supplies the authoritative reference to carry it through. Together they cut the guesswork at both ends: the Skill keeps the assistant out of the wrong product area, and the MCP Server fills in the specifics once the work gets detailed.

That pairing also trims a lot of back-and-forth. Without a Skill, an agent can burn several tool calls just working out which API it should be using. With the right Skill loaded, those MCP calls go towards real implementation questions instead of correcting the assistant's starting assumptions.

There's a discoverability angle here too. In a recent customer survey, roughly a third of respondents didn't know the DevExpress MCP Server existed. Skills give us a natural way to reintroduce it as part of a broader AI-assisted workflow, rather than leaving it as a separate tool people may never stumble across.


Why Smaller Models Benefit Even More

The big frontier models are impressive even when their product knowledge is patchy. Claude Opus, Claude Sonnet and the latest GPT-class models can often reason their way to something that works, especially with documentation access or a patient developer steering. They still slip up, but they tend to recover.

Smaller and older models have a harder time. Compact models like Claude Haiku and Gemini Flash (and the local models more teams are running now) are built to be fast, responsive and cheap. They're a great fit for plenty of coding work, but they carry less specialised product knowledge out of training.

This is where Skills earn their keep. Rather than asking a compact model to reverse-engineer an unfamiliar API from a handful of examples, the Skill hands it the essentials up front: current namespaces, recommended entry points, the usual patterns and the traps worth avoiding, all dropped straight into its working context.

And it matters beyond the hosted services. More teams are looking at local models for privacy, compliance or cost reasons. Those models will keep improving, but they're never going to hold detailed knowledge of every commercial framework or every recently shipped API. Skills close that gap by supplying the DevExpress-specific context regardless of whether the model runs in the cloud or on the developer's own machine.

The aim isn't to make a small model behave like a large one. It's to let you choose the model that suits your workflow without giving up confidence that it understands the DevExpress APIs you're working with.


Skills Versus Custom Instructions

Most AI assistants already give you a way to shape their behaviour. Copilot has custom instructions, Cursor has rule files, Claude Code has project instructions, and just about every other agent has its own flavour. Useful, all of them, but also fragmenting: each one wants its guidance in a different format.

Work across a few tools and you end up maintaining the same advice in several places at once. Coding conventions, preferred APIs, project-specific practices, all copied between config files and quietly drifting out of sync.

Skills take a more portable route. They load when they're relevant and stay out of the way when they're not: a Reporting Skill turns up for reporting work, a Barcode Skill for barcode generation, spreadsheet guidance when you're in Excel territory. The assistant gets what it needs without dragging a giant instruction file into every unrelated conversation.

That on-demand behaviour is the point. Loading everything into every request eats context and pulls the assistant's attention off the task at hand. Progressive disclosure keeps the working context tight, which really shows when you've got several DevExpress products installed, or a project that mixes reporting, document processing and web UI.

Portability is just as valuable. Because Skills aren't tied to one assistant, the same guidance travels across every supported agent. And as plugin-based distribution matures, Skills give updates a path to evolve alongside DevExpress products, instead of leaving each developer to hand-maintain their own instruction files.

The takeaway is simple: you shouldn't have to teach every AI assistant how to use DevExpress from scratch. Install the relevant Skills, pair them with the MCP Server when you need more depth, and let the tooling carry consistent guidance wherever the work happens.


Looking Beyond the Prompt

A year ago, most of the conversation around AI-assisted development was about prompt engineering, with whole posts hunting for the magic phrasing that would unlock better code. Prompts matter, no argument. But they're only part of the story. An assistant can only work with what it knows, and no amount of elegant wording makes up for knowledge that isn't there.

That's the lesson that runs through the team's work on the Office File API and Reporting Skills. When the assistant only half-knew DevExpress, it papered over the gaps with educated guesses. When it started from accurate context, the output was more reliable, more current, and a lot closer to what an experienced DevExpress developer would actually write.

The MCP Server rounds out the picture, handing compatible agents the current documentation, examples and API details whenever the task needs more depth. Skills get the assistant to the right starting point; the MCP Server keeps it honest from there.

Assistants will keep getting better. New models, stronger reasoning, today's rough edges smoothing out. What won't change is the value of context. General-purpose models will always do better when they understand the frameworks they're building on, especially frameworks that move faster than any training run can keep up with.

That's the thinking behind both DevExpress AI Skills and the DevExpress MCP Server. Between them, the assistant gets immediate guidance and access to current product knowledge. The payoff isn't just cleaner code. It's a development experience that feels less like correcting a stranger and more like working alongside someone who already knows the framework you're using.


Further Reading

Read the whole story
alvinashcraft
1 minute ago
reply
Pennsylvania, USA
Share this story
Delete

Things That Caught My Attention Last Week - July 6

1 Share

caught-my-i

Open-source

How GitHub maintains compliance for open source dependencies by The GitHub Blog

6 security settings every GitHub maintainer should enable this week by The GitHub Blog

Software Architecture

Addition by subtraction in software design by Oskar Dudycz

.NET

Validation in FastEndpoints: Pipelines, Pitfalls, and Patterns by Barret Blake

Closed class hierarchies by Andrew Lock

Visual Studio June Update - Track Your Usage, Trust Your Tools by Visual Studio Blog

A Big Week for the Critter Stack by Jeremy D. Miller

MCP Beyond the Chat Window: Build Diagnostics in CI by .NET Team

5 EF Core Performance Anti-Patterns That Entity Framework Extensions Eliminates by Chris Woody Woodruff

Add vs AddRange in EF Core: The Performance Myth You Need to Stop Repeating by Chris Woody Woodruff

Build Your Own VPN With Tailscale by Milan Jovanovic

EF Core UseIdentityColumns by Karen Payne

Software Design

Worse is better: JSON versus XML by Mark Seemann

REST/APIs

Query Is Here by Alexander Karan

Bringing Legacy Standards Into the Modern API Age by Chris Woody Woodruff

API Governance Is 75% People Work by Karen Payne

Azure

Upcoming Change: NTLM Removal in Git (libcurl) by Azure DevOps Blog

Azure SDK Release (June 2026) by Azure SDK Blog

Need a different partition key in Azure Cosmos DB? Pick the right approach by Azure Cosmos DB Blog

See our new Azure Cosmos DB Design Patterns by Azure Cosmos DB Blog

Software Development

The Plight of Women in Tech 20+ Years Later by Sarah Dutkiewicz

Read the whole story
alvinashcraft
1 minute ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories