In today’s episode, Cassidy, Marlene, and GPS tackle the ever-growing pile of new AI and agent terminology that's popped up seemingly overnight. They break down loop engineering (designing systems that prompt your agents instead of prompting them yourself), the Ralph loop that helped popularize it, and how squads and fleets let multiple agents split up planning, testing, and review work in parallel. They also dig into harness engineering (the scaffolding around a model that turns raw text generation into something useful), the newly coined “hill climbing,” and why forward deployed engineer might just be a solutions architect with better AI tooling. They wrap with a look at the spectrum between closed, open weight, and open source models, plus each host's open source pick of the week: Astro 7, the RAE API project, and Tau.
A researcher at a pharmaceutical company deploys a productivity assistant built on a well-regarded open-weight model. The base model is clean—it's been evaluated, its weights are public, its behavior is understood. A low-rank adaptation (LoRA) adapter is added to tune it for the lab's domain: it understands drug-discovery terminology, knows the team's workflows, and uses the right tools. The assistant is useful. Nobody looks too hard at the adapter file. Why would they?
Nowadays, everyone talks about AI. Building with AI, integrating AI into your application, and similar topics are hot topics. In today's post, I'll step into the .NET AI world by building a very simple Retrieval-Augmented Generation (RAG) implementation.
As it is our baby step, I intentionally designed a beginner-friendly introduction rather than a production-ready RAG implementation. We won't build a vector database or use embeddings yet. Instead, we'll use simple keyword matching to simulate the retrieval process and focus on understanding how the pieces of a RAG system fit together.
In future posts, we will climb the Everest of .NET AI, but for today sticking to the baby steps. Our idea is that instead of asking an AI model to answer a question using only what it already knows, we first retrieve relevant information from our own knowledge base and provide that information to the model as context.
What is Retrieval-Augmented Generation (RAG)?
RAG is an architecture for optimizing the performance of an artificial intelligence (AI) model by integrating it with external knowledge bases. RAG allows AI models to combine their powerful capabilities with a specific domain's or an organization's internal knowledge base to generate more relevant and accurate responses.
No matter how great an interior designer is, he cannot give you great design by just looking from the outside and not getting information about your home from inside. He needs to know what rooms look like and how they are placed. If you do not give him this information, he will make assumptions or give a very generic solution that sometimes proves wrong. A Large Language Model (LLM) works the same way, no matter how powerful Claude, ChatGPT, or any other model is, if you want it to work on your project, it needs its information. RAG basically combines AI's power with the organization's internal data to utilize it properly. Without the context, an AI model may rely on assumptions and generate information that may look credible but is not supported by your data. This is commonly known as an AI hallucination.
RAG helps AI propose concrete answers based on the data, otherwise, it can fall into guessing and lead to hallucinations.
Image by AWS
So RAG works in 3 steps: Retrieve ➡️ Augment ➡️ Generate
We retrieve relevant information from our knowledge base, add it to the AI prompt as context, and then let the AI model generate an answer based on that information.
Starting with RAG in an ASP.NET Core API
To take our first baby step, let's design an API with a simple RAG architecture.
Step 1: Create Api project
dotnet new webapi -n RAGPlayground
Step 2: Create an input model
namespace RAGPlayground.Models;
public class QueryInp
{
public string Question { get; set; } = string.Empty;
}
Step 3: Add a document store
The document store will represent our knowledge base, exposing a method called Search.
public interface IDocumentStore
{
List<string> Search(string query);
}
Its implementation is as follows:
public class DocumentStore : IDocumentStore
{
private readonly List<string> _documents =
[
"Minimal APIs simplify endpoint development in ASP.NET Core.",
"Entity Framework Core streamlines database operations.",
"Background services process long-running tasks efficiently.",
"Dependency injection improves application maintainability."
];
public List<string> Search(string query)
{
return _documents
.Select(doc => new
{
Document = doc,
Score = CalculateScore(doc, query)
})
.OrderByDescending(x => x.Score)
.Where(x => x.Score > 0)
.Take(3)
.Select(x => x.Document)
.ToList();
}
private int CalculateScore(string doc, string query)
{
var score = 0;
var queryWords = query
.ToLower()
.Split(' ', StringSplitOptions.RemoveEmptyEntries);
foreach (var word in queryWords)
{
if (doc.ToLower().Contains(word))
{
score++;
}
}
return score;
}
}
DocumentStore is a simple in-memory document repository that simulates the retrieval part of a RAG system. It stores a collection of documents and returns the most relevant ones based on keyword matching.
The _documents list contains the collection on which search is performed. Search involves four steps:
Calculate a relevance score for every document.
Sort documents from highest score to lowest.
Remove documents with a score of zero.
Return the top three matching documents.
The CalculateScore method converts the query string to lowercase before splitting it into individual words. Then it checks if each word appears in the document. The more query words found in the document, the higher the score becomes for that document.
Step 4: Create RAG layer
public interface IRagService
{
Task<string> AskAsync(string question);
}
The service implementation.
using RAGPlayground.Contracts;
namespace RAGPlayground.Services;
public class RagService: IRagService
{
private readonly IDocumentStore _documentStore;
public RagService(IDocumentStore documentStore)
{
_documentStore = documentStore;
}
public async Task<string> AskAsync(string question)
{
var context = _documentStore.Search(question);
if (context is null || !context.Any())
{
return "No relevant information found.";
}
var prompt = $"""
Based only on the context below, answer the user's question.
If the answer cannot be found in the context, say:
"I don't have enough information."
Context:
{context.First()}
Question:
{question}
""";
return await Task.FromResult(prompt);
}
}
The service layer is much simpler. It calls the underlying document store layer and returns the document with the highest score. The prompt variable concatenates context and question, mimicking the augmentation step of the RAG.
Step 5: Add a controller
using Microsoft.AspNetCore.Mvc;
using RAGPlayground.Contracts;
using RAGPlayground.Models;
namespace RAGPlayground.Controllers;
[ApiController]
[Route("api/[controller]")]
public class RagController : ControllerBase
{
private readonly IRagService _ragService;
public RagController(IRagService ragService)
{
_ragService = ragService;
}
[HttpPost("ask")]
public async Task<IActionResult> Ask([FromBody] QueryInp input)
{
var answer = await _ragService.AskAsync(input.Question);
return Ok(new { answer });
}
}
Now, I am exposing everything to the user via an ask endpoint.
Step 6: Run the project
dotnet run
In Postman, we can test the ask endpoint:
If a word is not found, a fitting answer is provided:
Step 7: Use simulation with an AI model
In production, you will use an AI such as OpenAI, Grok, Azure OpenAI, etc. In that case, the service layer method will include a contextual prompt. The AI client service is
public interface IAiClient
{
Task<string> GenerateAsync(string prompt);
}
Its implementation is as follows:
public class AiClient : IAiClient
{
public Task<string> GenerateAsync(string prompt)
{
// Call your actual AI provider here.
// For example: OpenAI, Azure OpenAI, Claude, Grok, etc.
throw new NotImplementedException(
"Connect an AI provider here.");
}
}
public class RagService: IRagService
{
private readonly IDocumentStore _documentStore;
private readonly IAiClient _aiClient;
public RagService(IDocumentStore documentStore, IAiClient aiClient)
{
_documentStore = documentStore;
_aiClient = aiClient;
}
public async Task<string> AskAsync(string question)
{
var context = _documentStore.Search(question);
if (context is null || !context.Any())
{
return "No relevant information found.";
}
var prompt = $"""
Based only on the context below, answer the user's question.
If the answer cannot be found in the context, say:
"I don't have enough information."
Context:
{context.First()}
Question:
{question}
""";
var answer = await _aiClient.GenerateAsync(prompt);
return answer;
}
}
The answer will be fetched via the AI client you are using. Finally, the answer is generated by an AI client, which we have abstracted for this article; in future posts, we will use a real AI client to go one step further. That last stage represents the generation part of the RAG.
What to consider while choosing RAG?
While RAG enhances the accuracy of AI models, you need to keep in mind a few points before going forward:
If the information your AI needs is very small and rarely changes, you can include it directly in the AI's system prompt instead of building a document search pipeline. Information like FAQs and product features is usually small enough to fit a few hundred tokens and send with every request. You don't need a RAG pipeline in such cases.
Structured questions such as how many customers onboarded last year or how much revenue our company generated are database queries rather than AI prompts. Consider function calling that lets the model query your APIs instead.
RAG is not good at answering questions that require analyzing an entire dataset, because RAG is designed to retrieve only the most relevant pieces, not every document. Don't use RAG for aggregation problems like finding this month's shopping trends or counting complaint categories.
A recommended approach to designing a RAG application is a layered approach, with prompt building, retrieval logic, and LLM integration residing in separate services.
To improve retrieval in a production-grade application, use embeddings instead of word matching. Once documents become embeddings, you need a vector database to store them.
Secure your API with JWT authentication and authorization to avoid information leaking to unauthorized users. Role-based Access Control (RBAC) is useful for restricting access to the application.
For a SaaS application, isolate tenants to better track and limit prompts for each client.
Conclusion
RAG empowers AI's accuracy by connecting it with system-specific knowledge. We saw how to use RAG architecture in an ASP.NET Core API using a basic knowledge base. For a real AI LLM integration, we will continue from here by implementing the AiClient class. Later, we list best practices to consider to use it effectively.
Spargine's ThreadPoolHelper simplifies concurrent execution in .NET applications by providing reusable APIs that manage thread pool operations. It supports single and batch operations, bounded concurrency, timeout management, cooperative cancellation, and non-throwing execution. By centralizing common patterns, ThreadPoolHelper enhances maintainability and reduces repetitive infrastructure code for developers.
When talking to developers I get wildly different opinions about how "good" GitHub Copilot actually was for them. Some love it, some find it slow, some weren't sure it was doing anything useful at all(just kidding). The problem: nobody had any data. We were talking about token usage, latency and tool calls purely from gut feeling.
GitHub Copilot can export all of that as OpenTelemetry traces, metrics and events. And the easiest way to look at it locally, without spinning up a cloud backend, is the Aspire Dashboard. Let's set that up.
Why do we need this?
Copilot isn't just autocomplete anymore. Every agent turn is a small orchestration: it calls a model, the model asks for tools, the tools run, the model answers. If you want to know where the time and the tokens go, you need to see that orchestration, not just the end result.
That's exactly what the Open Telemetry integration gives you. Every agent interaction produces a span tree:
Remark: Open Telemetry is off by default. No data is emitted, and the SDK isn't even loaded, until you explicitly enable it. Nothing to worry about on the privacy side unless you turn it on yourself.
Getting the Aspire Dashboard running
There are multiple ways to collect and visualize the Open Telemetry data. An easy way in is the Aspire Dashboard. It is a single container with a built-in OTLP endpoint and a trace viewer. No Azure subscription, no collector to configure. Run it with the Aspire CLI:
aspire dashboard run
Or, if you'd rather run the container image directly:
enabled turns telemetry on. captureContent is optional — it makes Copilot include the actual prompt and response text in the spans, instead of just token counts and durations. Handy while you're exploring locally, less handy on a shared machine.
Reload the window (settings changes don't apply to an already-running one), then have a normal chat conversation with Copilot. Ask it to read a file, run a command, whatever you'd do anyway.
Tip: enabling the setting alone doesn't emit anything. You need to actually send a chat message, and OTel batches before sending, so give it 10-30 seconds after your chat turn before you go looking in the dashboard.
Reading the traces
Back in http://localhost:18888, go to Traces.
You should see a trace per agent turn, named invoke_agent copilot (or invoke_agent claude if you're running Claude agent sessions through the Copilot Chat extension). Expand one and you get the full breakdown: how long the model call took, which tools ran, how long each of those took.
This is where the "Copilot feels slow" complaints get an actual answer. Sometimes it's the model call. Sometimes it's a tool — a slow runCommand, an MCP server doing something expensive — hiding behind what looks like one long agent turn.
The spans carry the GenAI semantic-convention attributes, so the same data works with any OTel backend later:
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens: token spend per call
gen_ai.usage.cache_read.input_tokens: how much you're benefiting from prompt caching
gen_ai.response.model: which model actually answered (Copilot silently reroutes sometimes)
gen_ai.tool.name: which tool ran inside execute_tool spans
Remark: if you're running the Copilot CLI in a separate terminal session rather than the VS Code chat panel, those traces show up as independent root traces under the github-copilot service instead of copilot-chat. They're not linked to your chat panel traces, but they land in the same dashboard, so filter by service.name if you want to tell them apart.
Going beyond gRPC vs HTTP
The default exporter is otlp-http, which is what the config above uses. If you'd rather use gRPC:
Remark: the Copilot CLI terminal runtime only speaks otlp-http, even if you configure gRPC. The Aspire Dashboard happens to serve both protocols on the same port, so this works transparently — you don't need two separate setups for chat-panel and CLI traces.
From local dashboard to team dashboard
The Aspire Dashboard is great for "let me quickly see what just happened on my machine," but it doesn't persist data across restarts and it's not meant for a whole team. Once you want aggregated numbers (token spend per team, cache hit rate over a sprint,...) you're looking at forwarding the same OTLP stream to Application Insights, Grafana, or another long-term backend through an OTel Collector.
Same settings, different endpoint. That's maybe a topic for another post.
For now: enable the setting, point it at a local Aspire Dashboard, and actually look at what your agent sessions are doing. Simple, useful, done.
“Workslop” is when your colleagues or bosses communicate with you by pasting big chunks of AI-generated text. The core problem with workslop is that the effort involved is asymmetrical, like a denial-of-service attack: it takes almost no effort to produce text with AI, but it still costs effort to read1. Here are some ways to protect yourself.
If you have enough authority or social capital, you can and should simply tell them “hey, don’t do that” (for instance, if you’re a senior engineer and an intern starts doing this to you). This is the easiest way to handle workslop. But you probably aren’t in a position to have that conversation with all of your colleagues, and you certainly can’t have it with everyone in your management chain.
One step above just telling a colleague to stop is to drive them around like a coding agent. I wrote about this in AI makes weak engineers less harmful: if a colleague is simply pasting your messages into Claude Code and sending you the outputs, you can treat them like a high-latency Slack interface to Claude Code. It won’t be as good as a normal coding agent, but it’ll often be better than nothing.
Another strategy is to use AI to fight AI. This is a good one for handling workslop from managers. You can do this in two broad ways. First, instead of carefully reading it, paste it into an LLM of your own and ask for a short list of the salient points. Second, you can sometimes simply ask an LLM for an entire response. In a sense, this makes you part of the problem, so I can see why some people might be uncomfortable with it. But it’s more sustainable than spending ten minutes of your effort for every ten seconds of theirs.
You can also bias toward calls or in-person meetings. Workslop is just a special case of the general “your coworker is bad at communication” problem. One classic way of handling this that works even better on AI content is to say “hey, let’s schedule some time to chat about it”. This works for two reasons: first, your colleagues can’t give you AI content over a call, and second, forcing people to spend a chunk of their time talking to you (i.e. to make the effort symmetrical) is a good way to filter out predators.
Finally, you can sometimes simply ignore the workslop. This is particularly true for long status updates or pull requests from outside of your organization2. You don’t have to respond to AI content as diligently as you would human content. You can match their lack of effort with your own: skim it, put off reading it until later (or never), and so on. If something’s really important, they’ll tell you in their own words.
Technically, not all cases of sending someone AI-generated content are workslop. If the effort is not asymmetrical — if the AI user has genuinely put a lot of their own time into the content — I don’t think it counts as slop, and you should just try and look past the AI style and treat it like a human message.
Some messages — particularly reports directed at the entire organization — may not be intended to be read at all. Written artifacts can have many purposes beyond communication: evidence of effort, a reference document for later communications, a way to cover somebody’s ass by proving they considered point X, something that can tick a compliance or process box, and so on. I wrote a lot more about this in Seeing like a software company.
Log in
