The System.Diagnostics.Process class is the primary way to create and interact with processes with .NET. We made the biggest update to it in years, with .NET 11. The changes add high-level APIs that make it easy to start a process and capture its output without deadlocks, give you full control over handle inheritance and standard handle redirection, introduce lifetime management features like KillOnParentExit, and include a lightweight SafeProcessHandle-based API surface that is more trimmer-friendly.

Summary

Here’s a recap of new Process APIs in .NET 11:

Other improvements include:

• Better scalability on Windows: BeginOutputReadLine/BeginErrorReadLine no longer block thread pool threads: throughput improvement when starting multiple processes in parallel with redirected output and error.
• Better trimmability: Up to 20% smaller NativeAOT binaries compared to .NET 10 when using Process and up to 32% smaller when using SafeProcessHandle.
• Improved process creation on Apple platforms: Up to 100x faster process creation on Apple Silicon due to switch to posix_spawn.
• Reduced memory allocations on Unix: 30–50% fewer allocations when starting processes on Unix.

The rest of the blog post is a deep dive into each of these features.

Capturing process output without deadlocks

Why capturing process output can hang your app

When redirecting standard output and error of the process, it’s possible to run into deadlocks. Knowing why it’s possible is crucial to understanding the changes we have made. Let’s build a C# app that tries to read all output and error from a process.

First of all, we need to redirect standard output and error of the process to be able to read it. This is done by setting RedirectStandardOutput and RedirectStandardError properties of ProcessStartInfo to true. Then before the process is started, two dedicated pipes are created (one for standard output and one for standard error), and the process is created with the write ends of these pipes (each pipe comes with a read and write end) as standard output and error. The child process just writes to its standard output and error as usual, but instead of going to the console, the data is written to the pipes.

ProcessStartInfo startInfo = new("dotnet", "--help")
{
    RedirectStandardOutput = true,
    RedirectStandardError = true
};

using Process process = new() { StartInfo = startInfo };

Pipes have limited buffer size (usually 4KB on Windows and 64KB on Unix). When the producer (in our case, the child process) writes to the pipe, the data is stored in a buffer until it’s read by the consumer (the parent process). If the producer writes more data than the size of the buffer and the consumer is not reading from the pipe at the same time, the producer will be blocked on write operation, waiting for the consumer to read from the pipe and free some space in the buffer.

If the consumer is waiting for the producer to exit (e.g. by calling WaitForExit) without reading from the pipe, it will be blocked as soon as the producer fills the buffer:

process.Start();
process.WaitForExit();

string output = process.StandardOutput.ReadToEnd();
string error = process.StandardError.ReadToEnd();

But does re-ordering the code help?

process.Start();

string output = process.StandardOutput.ReadToEnd();
string error = process.StandardError.ReadToEnd();

process.WaitForExit();

Not really. ReadToEnd is a blocking call – it reads until the stream reaches EOF, which only happens when the child process closes its end of the pipe (typically on exit). So in the code above, we first block on standard output until the child exits, and only then start reading standard error. While we’re waiting on standard output, nobody is draining standard error. If the child writes more to stderr than the pipe buffer can hold, the child blocks on its write – and we’re stuck waiting for each other.

The root cause is that we are reading the two streams sequentially, not simultaneously. To avoid this deadlock, we need to drain both standard output and error at the same time. So far, we had two options:

The existing APIs are not optimal in terms of simplicity and performance. I’ll show you a couple patterns.

Use asynchronous read operations on StandardOutput and StandardError

The Process class exposes stream readers that you can read from, for example, with ReadToEndAsync.

process.Start();

// Start both operations to ensure both streams are drained at the same time
Task<string> outputTask = process.StandardOutput.ReadToEndAsync();
Task<string> errorTask = process.StandardError.ReadToEndAsync();

// Wait for both read operations to complete and process to exit
await Task.WhenAll(outputTask, errorTask, process.WaitForExitAsync());

string output = await outputTask;
string error = await errorTask;

Use OutputDataReceived and ErrorDataReceived events

These Process class events are raised when a line is written to standard output and error respectively.

StringBuilder stdOut = new(), stdErr = new();

process.OutputDataReceived += (sender, e) => stdOut.AppendLine(e.Data);
process.ErrorDataReceived += (sender, e) => stdErr.AppendLine(e.Data);

process.Start();

process.BeginOutputReadLine();
process.BeginErrorReadLine();

process.WaitForExit();

Process.ReadAllText and Process.ReadAllTextAsync

We have added ReadAllText and ReadAllTextAsync (PR) methods to Process class. They drain standard output and error at the same time, helping us to avoid deadlocks. They decode the output using the encoding specified by the ProcessStartInfo.Standard[Output/Error]Encoding (or the default when not specified), and return the result as strings (per-line needs are handled by an API we’ll see shortly).

public class Process
{
    public (string StandardOutput, string StandardError) ReadAllText(TimeSpan? timeout = default);
    public Task<(string StandardOutput, string StandardError)> ReadAllTextAsync(CancellationToken cancellationToken = default);
}

So the code to read all output and error from the process becomes much simpler:

ProcessStartInfo startInfo = new("dotnet", "--help")
{
    RedirectStandardOutput = true,
    RedirectStandardError = true
};

using Process process = new() { StartInfo = startInfo };
process.Start();

(string output, string error) = process.ReadAllText();
process.WaitForExit();

Process.RunAndCaptureText and Process.RunAndCaptureTextAsync

We expect that capturing output and error and just waiting for the process to exit (process can close standard handles and keep running) is very common. That is why we have introduced RunAndCaptureText and RunAndCaptureTextAsync methods that combine starting the process, reading all output and error, and waiting for the process to exit in one method call.

namespace System.Diagnostics;

public sealed class ProcessExitStatus
{
    public ProcessExitStatus(int exitCode, bool canceled, PosixSignal? signal = null);
    public int ExitCode { get; }
    public PosixSignal? Signal { get; }
    public bool Canceled { get; }
}

public sealed class ProcessTextOutput
{
    public ProcessTextOutput(ProcessExitStatus exitStatus, string standardOutput, string standardError, int processId);
    public ProcessExitStatus ExitStatus { get; }
    public string StandardOutput { get; }
    public string StandardError { get; }
    public int ProcessId { get; }
}

public class Process
{
    public static ProcessTextOutput RunAndCaptureText(ProcessStartInfo startInfo, TimeSpan? timeout = default);
    public static ProcessTextOutput RunAndCaptureText(string fileName, IList<string>? arguments = null, System.TimeSpan? timeout = default);

    public static Task<ProcessTextOutput> RunAndCaptureTextAsync(ProcessStartInfo startInfo, CancellationToken cancellationToken = default);
    public static Task<ProcessTextOutput> RunAndCaptureTextAsync(string fileName, IList<string>? arguments = null, CancellationToken cancellationToken = default);
}

Which makes the code to start a process and capture its output and error literally a one liner:

ProcessTextOutput output = Process.RunAndCaptureText("dotnet", ["--help"]);

We also provided Process.Run and Process.RunAsync methods that don’t capture output and error, but just wait for the process to exit, for cases when you don’t care about the output.

ProcessExitStatus status = Process.Run("dotnet", ["build", "-c", "Release"]);

Process.ReadAllLines and Process.ReadAllLinesAsync

If you need to capture output and error as lines, you can use ReadAllLines (PR) and ReadAllLinesAsync (PR) methods, which are implemented in the same way as their ReadAllText counterparts, but return an enumerable of ProcessOutputLine instead of a single string.

namespace System.Diagnostics;

public readonly struct ProcessOutputLine
{
    public ProcessOutputLine(string content, bool standardError);
    public string Content { get; }
    public bool StandardError { get; }
}

public class Process
{
    public IEnumerable<ProcessOutputLine> ReadAllLines(TimeSpan? timeout = default);
    public IAsyncEnumerable<ProcessOutputLine> ReadAllLinesAsync(CancellationToken cancellationToken = default);
}

Let’s see how we can use it to read output and error from the process line by line as they are produced:

using Process process = Process.Start("dotnet", "--help")!;
await foreach (ProcessOutputLine line in process.ReadAllLinesAsync())
{
    if (line.StandardError)
        Console.ForegroundColor = ConsoleColor.Red;

    Console.WriteLine(line.Content);
    Console.ResetColor();
}

Process.ReadAllBytes and Process.ReadAllBytesAsync

If you need to capture output and error as bytes, you can use ReadAllBytes method, which is internally used by ReadAllText (PR). It returns byte arrays instead of strings.

public class Process
{
    public (byte[] StandardOutput, byte[] StandardError) ReadAllBytes(TimeSpan? timeout = default);
    public Task<(byte[] StandardOutput, byte[] StandardError)> ReadAllBytesAsync(CancellationToken cancellationToken = default);
}

Timeouts and cancellation

All the aforementioned methods that read from standard output and error support timeouts and cancellation. If the timeout is reached or the cancellation token is cancelled before the end of the stream is reached, the methods will throw TimeoutException or OperationCanceledException respectively. The high-level RunAndCaptureText[Async] and Run[Async] methods will also try to kill the process to avoid leaving it running.

Multiplexing and other optimizations under the hood

The new methods are not just simpler to use, but also faster. Behind the scenes, the synchronous Process.RunAndCaptureText and Process.ReadAll[Bytes/Text] methods use multiplexing (poll on Unix and WaitForMultipleObjects on Windows) to read from both standard output and error using a single thread. They also implement a bunch of other optimizations such as using ArrayPool to reduce memory allocations. The asynchronous Process.RunAndCaptureTextAsync and Process.ReadAllTextAsync methods use asynchronous I/O operations without blocking any threads.

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Diagnostics;
using System.Text;

BenchmarkSwitcher.FromAssembly(typeof(CaptureOutputBenchmarks).Assembly).Run(args);

[MemoryDiagnoser, ThreadingDiagnoser]
public class CaptureOutputBenchmarks
{
    private readonly ProcessStartInfo _processStartInfo = CreateStartInfo();

    private static ProcessStartInfo CreateStartInfo()
    {
        ProcessStartInfo startInfo = OperatingSystem.IsWindows()
            ? new("cmd.exe", "/c for /L %i in (1,1,1000) do @echo Line %i")
            : new("sh", ["-c", "for i in $(seq 1 1000); do echo \"Line $i\"; done"]);

        startInfo.RedirectStandardOutput = true;
        startInfo.RedirectStandardError = true;

        return startInfo;
    }

    [Benchmark]
    public int Events()
    {
        using Process process = new();
        process.StartInfo = _processStartInfo;

        StringBuilder stdOut = new(), stdErr = new();

        process.OutputDataReceived += (sender, e) => stdOut.AppendLine(e.Data);
        process.ErrorDataReceived += (sender, e) => stdErr.AppendLine(e.Data);

        process.Start();

        process.BeginOutputReadLine();
        process.BeginErrorReadLine();

        process.WaitForExit();

        // Other benchmarks materialize the output, so we do it here
        // to ensure it's apples to apples comparison.
        _ = stdOut.ToString();
        _ = stdErr.ToString();

        return process.ExitCode;
    }

    [Benchmark]
    public async Task<int> ReadToEndAsync()
    {
        using Process process = Process.Start(_processStartInfo)!;

        Task<string> readOutput = process.StandardOutput.ReadToEndAsync();
        Task<string> readError = process.StandardError.ReadToEndAsync();

        _ = await readOutput;
        _ = await readError;

        await process.WaitForExitAsync();

        return process.ExitCode;
    }

    [Benchmark]
    public int RunAndCaptureText()
    {
        ProcessTextOutput processTextOutput = Process.RunAndCaptureText(_processStartInfo);

        _ = processTextOutput.StandardOutput;
        _ = processTextOutput.StandardError;

        return processTextOutput.ExitStatus.ExitCode;
    }

    [Benchmark]
    public async Task<int> RunAndCaptureTextAsync()
    {
        ProcessTextOutput processTextOutput = await Process.RunAndCaptureTextAsync(_processStartInfo);

        _ = processTextOutput.StandardOutput;
        _ = processTextOutput.StandardError;

        return processTextOutput.ExitStatus.ExitCode;
    }
}

BenchmarkDotNet v0.16.0-nightly.20260505.517, Windows 11 (10.0.26200.8246/25H2/2025Update/HudsonValley2)
AMD Ryzen Threadripper PRO 3945WX 12-Cores 3.99GHz, 1 CPU, 24 logical and 12 physical cores
Memory: 63.86 GB Total, 44.02 GB Available
.NET SDK 11.0.100-preview.5.26255.101
  [Host]     : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3
  DefaultJob : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3

// * Legends *
  Completed Work Items : The number of work items that have been processed in ThreadPool (per single operation)
  Allocated            : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)

As you can see, on Windows the synchronous RunAndCaptureText method is about 2-3 ms faster than the old approaches and allocates about 4.5x less memory. It also doesn’t use thread pool at all.

BenchmarkDotNet v0.16.0-nightly.20260505.517, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
AMD Ryzen Threadripper PRO 3945WX 12-Cores 3.99GHz, 1 CPU, 24 logical and 12 physical cores
Memory: 31.27 GB Total, 29.69 GB Available
.NET SDK 11.0.100-preview.5.26255.101
  [Host]     : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3
  DefaultJob : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3

On Linux, the new methods allocate about 2-4x less memory and the synchronous method also does not use thread pool at all.

Handle inheritance

In case of pipes, the end of the file (EOF) is reached when all handles to its write end have been closed. Process closes its own copy of the write end of the pipe after starting the process, but in order for the pipe to be used by the child process, it must be inherited, which requires the pipe to be inheritable. When a process is started, it by default inherits all inheritable handles from the parent process, which opens the door for another two issues that can lead to deadlocks:

• when a sibling process started concurrently inherits the pipe handle and keeps it open,
• when a grandchild process inherits the pipe handle and keeps it open after the child process exits.

From .NET perspective, there is no way to prevent this kind of accidental handle inheritance by the grandchild process, as the child process can do whatever it wants with the handles it inherits from the parent. The best we can do is to provide an API that allows the child process to inherit only selected handles. And this is exactly what we have done by introducing ProcessStartInfo.InheritedHandles property (PR) that allows you to specify which handles should be inherited by the child process.

public class ProcessStartInfo
{
    public IList<SafeHandle>? InheritedHandles { get; set; } = null;
}

Due to backwards compatibility reasons, the new property is set to null by default, which means that the behavior is the same as before – all inheritable handles are inherited. If you set it to an empty list, only standard handles will be inherited. If you set it to a list of specific handles, those handles will be inherited along with the standard handles.

Request for feedback: we are considering extending all the new APIs that capture process output with an ability to stop when the process exits but the pipes remain open. Please let us know if you are interested in this feature.

Important: handles in this list should not have inheritance enabled beforehand. If they do, they could be unintentionally inherited by other processes started concurrently with different APIs, which may lead to security or resource management issues.

Note: as of today only SafeFileHandle and SafePipeHandle are supported, if you need more, please let us know.

Performance implications: if InheritedHandles is not null:

In short: as long as you don’t run on old Linux kernels (prior to 5.9), there should be no performance regression compared to the old behavior when inheriting all handles.

• On Windows we acquire only a reader lock when starting new process. It means that if you are starting multiple processes in parallel and they are all setting InheritedHandles, they won’t be blocked on process creation as they would be if we used a global lock.
• On Unix, we use the best available sys-call to ensure that only the specified handles are inherited by the child process:
  • On Apple platforms, we always use posix_spawn with POSIX_SPAWN_CLOEXEC_DEFAULT flag, which is supported by all versions supported by current .NET.
  • On Linux, we use close_range or __NR_close_range if they are available and enabled.
  • On FreeBSD we use close_range (available since FreeBSD 12.2).
  • On Illumos/Solaris, we use fdwalk.
  • If none of the above is available (or enabled), we fallback to iterating over all file descriptors and setting FD_CLOEXEC manually. This is expensive and can cause major performance regression. Mostly for Linux kernels prior to 5.9 (except RHEL 8.0 which backported it).

Let’s benchmark the performance implications!

public class GlobalLock
{
    private ProcessStartInfo info;

    [Params(true, false)]
    public bool SetInheritedHandles { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        info = OperatingSystem.IsWindows()
            ? new("cmd.exe", ["/c", "exit 42"])
            : new("sh", ["-c", "exit 42"]);

        info.InheritedHandles = SetInheritedHandles ? [] : null;
    }

    [Benchmark]
    public ParallelLoopResult Run() => Parallel.For(0, 1_000, (_, _) => _ = Process.Run(info));
}

We can see that on this Windows machine, the throughput has doubled:

BenchmarkDotNet v0.16.0-nightly.20260505.517, Windows 11 (10.0.26200.8246/25H2/2025Update/HudsonValley2)
AMD Ryzen Threadripper PRO 3945WX 12-Cores 3.99GHz, 1 CPU, 24 logical and 12 physical cores
Memory: 63.86 GB Total, 32.5 GB Available
.NET SDK 11.0.100-preview.5.26255.101
  [Host] : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3
  Dry    : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3

InvocationCount=1  IterationCount=10  LaunchCount=1
UnrollFactor=1  WarmupCount=1

Redirecting standard handles

Another way to limit handle inheritance issues is to let the users redirect standard handles to any file handle they want, without the need to make it inheritable. It’s now possible to redirect standard handles to any file handle (PR), which opens up new scenarios such as:

• process piping,
• redirecting to a file,
• redirecting to a null handle (it reports 0 bytes read/written for every operation):
  • starting process with no input,
  • discarding output,
• starting process with async handles (advanced, niche scenario),
• breaking inheritance chain by redirecting standard handles to other handles than parent process handles.

The new API comes with a few new enablers (File.OpenHandle already existed) that make it easier to use:

namespace System.Diagnostics
{
    public class ProcessStartInfo
    {
        public SafeFileHandle? StandardInputHandle { get; set; }
        public SafeFileHandle? StandardOutputHandle { get; set; }
        public SafeFileHandle? StandardErrorHandle { get; set; }
    }
}

namespace Microsoft.Win32.SafeHandles
{
    public class SafeFileHandle
    {
        public static SafeFileHandle CreateAnonymousPipe(out SafeFileHandle readPipe, out SafeFileHandle writePipe, 
            bool asyncRead = false, bool asyncWrite = false);
    }
}

namespace System.IO
{
    public static class File
    {
        public static SafeFileHandle OpenHandle(string path, FileMode mode = FileMode.Open, FileAccess access = FileAccess.Read, FileShare share = FileShare.Read, FileOptions options = FileOptions.None, long preallocationSize = 0);
        public static SafeFileHandle OpenNullHandle();
    }
}

namespace System
{
    public static class Console
    {
        public static SafeFileHandle OpenStandardInputHandle();
        public static SafeFileHandle OpenStandardOutputHandle();
        public static SafeFileHandle OpenStandardErrorHandle();
    }
}

Let’s pipe ls /usr/bin into grep zip and redirect the output to a file to find zip-related commands:

ls /usr/bin | grep zip > output.txt

And now we’re going to implement the same thing in C# with the new APIs.

SafeFileHandle.CreateAnonymousPipe(out SafeFileHandle readPipe, out SafeFileHandle writePipe);

using (readPipe)
using (writePipe)
using (SafeFileHandle outputFile = File.OpenHandle("output.txt", FileMode.Create, FileAccess.Write))
{
    ProcessStartInfo producer = new("ls", ["/usr/bin"])
    {
        StandardOutputHandle = writePipe
    };

    // Start consumer with input from the read end of the pipe, writing output to file
    ProcessStartInfo consumer = new("grep", ["zip"])
    {
        StandardInputHandle = readPipe,
        StandardOutputHandle = outputFile,
    };

    using Process producerProcess = Process.Start(producer)!;
    // The producer process has its own copy of the write end of the pipe, we need to dispose the parent copy.
    writePipe.Dispose();

    using Process consumerProcess = Process.Start(consumer)!;
    // The consumer process has its own copy of the read end of the pipe, we need to dispose the parent copy.
    readPipe.Dispose();

    await producerProcess.WaitForExitAsync();
    await consumerProcess.WaitForExitAsync();
}

Note: all of the new enablers create non-inheritable handles but Process knows how to make them inheritable when starting the process, so you don’t have to worry about handle inheritance at all.

Other SafeFileHandle improvements

It’s worth noting that SafeFileHandle class has also received some new features that make it easier to work with:

• Type property to check if the handle type is a file, pipe, console, etc (PR).
• IsAsync returns true on Unix only if the handle has O_NONBLOCK flag enabled.
• All read and write RandomAccess methods now support non-seekable handles (PR) such as pipes, which makes it possible to use them without the need to use FileStream.

namespace System.IO
{
    public enum FileHandleType
    {
        Unknown = 0,
        RegularFile,
        Pipe,
        Socket,
        CharacterDevice,
        Directory,
        SymbolicLink,
        BlockDevice,
    }
}

namespace Microsoft.Win32.SafeHandles
{
    public class SafeFileHandle
    {
        public FileHandleType Type { get; }
    }
}

Lifetime management

Process.StartAndForget

There is a common misconception that when a process is disposed, it’s also being killed. This is not the case, as Process.Dispose only releases the resources associated with the process, but does not kill it.

To make it easier to start a process without the need to worry about disposing it, we have introduced Process.StartAndForget method that starts a process, returns its ID and releases all resources associated with it (PR).

public class Process
{
    public static int StartAndForget(ProcessStartInfo startInfo);
    public static int StartAndForget(string fileName, IList<string>? arguments = null);
}

The usage is straightforward:

int processId = Process.StartAndForget("notepad.exe");

ProcessStartInfo.KillOnParentExit

Processes started by the parent process are not automatically terminated when the parent process exits. This can lead to orphaned processes that keep running in the background, which is not desirable in many scenarios. To address this issue, we have introduced ProcessStartInfo.KillOnParentExit property that ensures that the child process is killed when the parent process exits (including forced terminations and crashes).

public class ProcessStartInfo
{
    [SupportedOSPlatform("windows")] // introduced in .NET 11 Preview 4
    [SupportedOSPlatform("linux")] // introduced in .NET 11 Preview 5
    [SupportedOSPlatform("android")] // introduced in .NET 11 Preview 5
    public bool KillOnParentExit { get; set; }
}

This is achieved by using platform-specific features such as JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE on Windows (PR) and PR_SET_PDEATHSIG on Linux and Android (PR). In contrast to other APIs, the behavior is slightly different on different platforms:

• On Windows, we need to use Job object to ensure that the child process is killed when the parent process exits. Job objects are by default inherited by all child processes, so if the child process spawns another process (a grandchild), that grandchild will also be terminated when the parent process exits.
• On Linux and Android, we use PR_SET_PDEATHSIG to specify a SIGKILL that the kernel will send to the child process when the thread that created the process exits. Since both Thread Pool and user threads can be terminated at any time, we maintain a dedicated thread used only for spawning processes with KillOnParentExit enabled, to ensure that the child processes are killed when the parent process exits. So when there are multiple processes started with KillOnParentExit, a synchronization mechanism is used to ensure that the dedicated thread spawns one process at a time.

Request for feedback: we are considering extending the API to support killing child processes when the parent process exits on other Unix platforms as well. Since none of them provides a similar mechanism, we could handle only normal (atexit) and graceful terminations (SIGTERM etc). If you are interested in this feature, please let us know.

ProcessStartInfo.StartDetached

ProcessStartInfo.StartDetached property allows you to start a process that is detached from the parent process, which means that it will keep running even if the parent process exits, gets signaled or terminal is closed. This is achieved by using platform-specific features such as DETACHED_PROCESS flag on Windows and setsid on Unix (PR).

public class ProcessStartInfo
{
    public bool StartDetached { get; set; }
}

Moreover, if StartDetached is set to true and no redirection for standard handles is specified, standard handles will be redirected to null handle to avoid keeping parent standard handles open unnecessarily.

SafeProcessHandle

Sometimes Process doesn’t cover your scenario – for example, you may need to P/Invoke CreateProcessAsUser on Windows or use a custom posix_spawn configuration on Unix. In those cases, you already have an OS process handle, but so far SafeProcessHandle has not provided any public APIs other than constructor. We’ve extended it with a set of focused APIs for the most common operations:

namespace Microsoft.Win32.SafeHandles
{
    public class SafeProcessHandle : SafeHandle
    {
        public int ProcessId { get; }
        public void Kill();
        public bool Signal(PosixSignal signal);
        public static SafeProcessHandle Start(ProcessStartInfo startInfo);
        public bool TryWaitForExit(System.TimeSpan timeout, [NotNullWhen(true)] out ProcessExitStatus? exitStatus);
        public ProcessExitStatus WaitForExit();
        public Task<ProcessExitStatus> WaitForExitAsync(CancellationToken cancellationToken = default);
        public Task<ProcessExitStatus> WaitForExitOrKillOnCancellationAsync(CancellationToken cancellationToken);
        public ProcessExitStatus WaitForExitOrKillOnTimeout(TimeSpan timeout);
    }
}

The Process class itself already exposes SafeProcessHandle via Process.SafeHandle property, so you can use the new APIs even if you are using Process class:

[UnsupportedOSPlatform("windows")] // SIGTERM is not supported on Windows
ProcessExitStatus TerminateProcess(Process process)
{
    // First try to terminate the process gracefully with SIGTERM
    process.SafeHandle.Signal(PosixSignal.SIGTERM);
    if (process.SafeHandle.TryWaitForExit(TimeSpan.FromSeconds(3), out ProcessExitStatus? exitStatus))
    {
        return exitStatus;
    }

    // If the process is still running after the timeout, kill it forcefully with SIGKILL
    process.SafeHandle.Signal(PosixSignal.SIGKILL);
    return process.SafeHandle.WaitForExit();
}

Or if you want to kill the process if it doesn’t exit within a certain timeout:

using SafeProcessHandle processHandle = SafeProcessHandle.Start(new ProcessStartInfo("myapp.exe"));
ProcessExitStatus exitStatus = processHandle.WaitForExitOrKillOnTimeout(TimeSpan.FromMinutes(1));
if (exitStatus.Canceled)
{
    Console.WriteLine("The process was killed after timeout.");
}

Trimmability

SafeProcessHandle also offers better trimmability. Let’s publish a NativeAOT app that starts a process and waits for it to exit.

Using SafeProcessHandle:

using SafeProcessHandle handle = SafeProcessHandle.Start(new ProcessStartInfo("whoami"));
handle.WaitForExit();

And Process:

using Process process = Process.Start(new ProcessStartInfo("whoami"))!;
process.WaitForExit();

dotnet publish -c Release -r win-x64 -p:PublishAot=true
dotnet publish -c Release -r linux-x64 -p:PublishAot=true

The size on disk improvements for Process (PR) include a community contribution from Red Hat. It’s worth noting that Tom Deseyn from Red Hat has contributed a LOT to this release by reviewing the Linux implementations of the new APIs, so a big thank you to him!

Notable performance improvements

Improved scalability on Windows

So far, both Process.BeginOutputReadLine and Process.BeginErrorReadLine were creating a background task that was performing blocking read on the output/error pipe. So for every process that was started with redirected output and error that used the Begin[Output/Error]ReadLine methods, two thread pool threads were being blocked (#81896). For years, we have believed that it’s impossible to solve this issue on Windows, as there is no support for truly asynchronous I/O operations on anonymous pipes.

But when reading the documentation, we have found out that:

“Anonymous pipes are implemented using a named pipe with a unique name. Therefore, you can often pass a handle to an anonymous pipe to a function that requires a handle to a named pipe.”

We knew that named pipes support truly asynchronous I/O operations on Windows and combined with our previous experience from File IO improvements in .NET 6 we knew that it’s possible to open one end of named pipe for asynchronous IO and the other end for synchronous IO (99.99% of applications expect standard handles to be opened for synchronous IO).

We studied the CreatePipe implementation and ensured (PR) that the new approach does not introduce any breaking changes. Starting from .NET 11 Preview 4, on Windows, when you start a process with redirected output and error, the pipes are created as named pipes with the read end opened for asynchronous IO and the write end opened for synchronous IO. This allows us to use truly asynchronous IO operations on the output and error pipes without blocking any threads.

Last but not least, we have exposed the ability to create anonymous pipes using SafeFileHandle.CreateAnonymousPipe method, which creates a pair of connected pipes and returns their handles. This method is available on both Windows and Unix, and it abstracts away the platform-specific details of creating pipes.

All of that translates into much better scalability when starting multiple processes in parallel with redirected output and error on Windows, as we are no longer blocking thread pool threads for every process.

public class BeginReadLineBenchmarks
{
    private static readonly ProcessStartInfo _processStartInfo = CreateStartInfo();

    private static ProcessStartInfo CreateStartInfo()
    {
        ProcessStartInfo startInfo = OperatingSystem.IsWindows()
            ? new("cmd.exe", "/c for /L %i in (1,1,1000) do @echo Line %i")
            : new("sh", ["-c", "for i in $(seq 1 1000); do echo \"Line $i\"; done"]);

        startInfo.RedirectStandardOutput = true;
        startInfo.RedirectStandardError = true;

        return startInfo;
    }

    [Benchmark]
    public ParallelLoopResult Run() => Parallel.For(0, 300, static (_, _) => _ = Events());

    private static int Events()
    {
        using Process process = new();
        process.StartInfo = _processStartInfo;

        StringBuilder stdOut = new(), stdErr = new();

        process.OutputDataReceived += (sender, e) => stdOut.AppendLine(e.Data);
        process.ErrorDataReceived += (sender, e) => stdErr.AppendLine(e.Data);

        process.Start();

        process.BeginOutputReadLine();
        process.BeginErrorReadLine();

        process.WaitForExit();

        return process.ExitCode;
    }
}

BenchmarkDotNet v0.16.0-nightly.20260505.517, Windows 11 (10.0.26200.8246/25H2/2025Update/HudsonValley2)
AMD Ryzen Threadripper PRO 3945WX 12-Cores 3.99GHz, 1 CPU, 24 logical and 12 physical cores
Memory: 63.86 GB Total, 39.4 GB Available
.NET SDK 11.0.100-preview.5.26255.101
  [Host]    : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3
  .NET 10.0 : .NET 10.0.7 (10.0.7, 10.0.726.21808), X64 RyuJIT x86-64-v3
  .NET 11.0 : .NET 11.0.0 (11.0.0-preview.5.26255.101, 11.0.26.25601), X64 RyuJIT x86-64-v3

As you can see, for this particular micro-benchmark and machine, the throughput has improved by about 1.8x. The improvement will be even more significant when starting more processes in parallel, as we are no longer blocking thread pool threads for every process.

Improved process creation on apple platforms

In order to implement ProcessStartInfo.InheritedHandles on apple platforms (macOS, MacCatalyst), we had to switch from fork + exec to posix_spawn (PR). To tell the long story short, it offers much better performance on apple platforms, especially on the arm64 architecture.

When running following benchmarks:

[MemoryDiagnoser]
public class ProcessStartBenchmarks
{
    private static ProcessStartInfo s_startProcessStartInfo = new ProcessStartInfo()
    {
        FileName = "whoami", // exists on both Windows and Unix, and has very short output
        RedirectStandardOutput = true // avoid visible output
    };

    private Process? _startedProcess;

    [Benchmark]
    public void Start()
    {
        _startedProcess = Process.Start(s_startProcessStartInfo)!;
    }

    [IterationCleanup(Target = nameof(Start))]
    public void CleanupStart()
    {
        if (_startedProcess != null)
        {
            _startedProcess.WaitForExit();
            _startedProcess.Dispose();
            _startedProcess = null;
        }
    }

    [Benchmark]
    public void StartAndWaitForExit()
    {
        using (Process p = Process.Start(s_startProcessStartInfo)!)
        {
            p.WaitForExit();
        }
    }
}

We have observed an impressive improvement of about 98x on Apple Silicon and about 4.5x on x64 machines:

BenchmarkDotNet v0.15.8, macOS Sequoia 15.4.1 (24E263) [Darwin 24.4.0]
Apple M4, 1 CPU, 10 logical and 10 physical cores
.NET SDK 11.0.100-preview.3.26174.112
  [Host]     : .NET 10.0.5 (10.0.5, 10.0.526.15411), Arm64 RyuJIT armv8.0-a

BenchmarkDotNet v0.15.8, macOS Sequoia 15.2 (24C101) [Darwin 24.2.0]
Intel Core i5-8500B CPU 3.00GHz (Coffee Lake), 1 CPU, 6 logical and 6 physical cores
.NET SDK 11.0.100-preview.3.26174.112
  [Host]     : .NET 10.0.5 (10.0.5, 10.0.526.15411), X64 RyuJIT x86-64-v3

Reduced memory allocations on Unix

We have also reduced memory allocation by 30-50% when starting process on Unix (PR).

BenchmarkDotNet v0.15.8, macOS Sequoia 15.4.1 (24E263) [Darwin 24.4.0]
Apple M2, 1 CPU, 8 logical and 8 physical cores
.NET SDK 11.0.100-preview.3.26178.103
  [Host]     : .NET 10.0.5 (10.0.5, 10.0.526.15411), Arm64 RyuJIT armv8.0-a

Call to Action

All of these improvements are available today in .NET 11 Preview 4. Give it a try, and let us know what you think – leave a comment here or file issues or feature requests at dotnet/runtime. We’d love your feedback!

The post Process API Improvements in .NET 11 appeared first on .NET Blog.

Learn T-SQL With Erik: Partitioning != Performance

Fair warning, the audio got pretty garbled on this one. If you don’t like space alien voices, hit mute and/or skip ahead to the transcript.

Chapters

• *00:00:00* – Introduction
• *00:01:32* – Course Information
• *00:02:19* – Personal Appearances
• *00:03:17* – Upcoming Events
• *00:04:11* – Data Management Feature
• *00:05:01* – Table Partitioning
• *00:05:58* – Batch Mode Limitations
• *00:06:33* – Query Execution
• *00:07:02* – Indexing Issues
• *00:08:04* – Column Wrapping
• *00:09:01* – Query Plan Analysis
• *00:10:22* – Non-Aligned Indexes
• *00:11:03* – Index Optimization
• *00:12:19* – Query Performance
• *00:13:15* – Time Comparison
• *00:14:00* – Query Rewriting
• *00:15:02* – Partitioning Challenges
• *00:16:00* – Data Swapping
• *00:16:45* – Conclusion
• *00:17:06* – Next Video

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. Blog readers get 25% off the Everything Bundle — over 100 hours of performance tuning content. Need hands-on help? I offer consulting engagements from targeted investigations to ongoing retainers. Want a quick sanity check before committing to a full engagement? Schedule a call — no commitment required.

The post Learn T-SQL With Erik: Partitioning != Performance appeared first on Darling Data.

Artificial Intelligence is evolving rapidly. What began as simple prompt-and-response systems is now transforming into fully autonomous, agentic AI architectures capable of reasoning, orchestrating tools, interacting with enterprise data, and invoking external systems dynamically. While these capabilities unlock enormous business potential, they also introduce an entirely new category of security challenges.

Organizations are no longer asking only:

“How do we build AI systems?”

They are now asking:

“How do we build AI systems securely, responsibly, and with governance built into every layer?”

This is where security architecture becomes critical.

Modern AI systems introduce threats that traditional applications were never designed to handle. Prompt injection, data poisoning, over-privileged agents, hidden data exfiltration, unauthorized tool execution, and lack of operational traceability are becoming real concerns as enterprises move toward production-scale AI adoption.

To address these emerging risks, MAESTRO framework is a layered threat modeling approach designed specifically for AI and Agentic AI systems. At the same time, Microsoft SQL introduces a powerful set of AI-enabled capabilities that bring AI closer to enterprise data while maintaining strong governance, observability, and security boundaries.

This combination creates an interesting architectural opportunity:

Microsoft SQL is no longer just a database. It becomes a governed execution boundary for enterprise AI systems.

Understanding the MAESTRO Framework

The MAESTRO framework provides a structured way to think about security risks across AI systems. Instead of viewing AI as a single application component, MAESTRO breaks the architecture into multiple operational layers, each with its own attack surface and security concerns.

These layers include:

• Foundation Models
• Data Operations
• Agent Frameworks
• Deployment & Infrastructure
• Evaluation & Observability
• Security & Compliance
• Agent Ecosystem

What makes MAESTRO particularly important is that it recognizes a fundamental shift in how applications behave. Traditional threat modeling frameworks such as STRIDE were designed around predictable application behavior, predefined execution paths, and relatively static trust boundaries. Agentic AI systems introduce a fundamentally different operating model. These systems operate dynamically at runtime, combining user input, retrieved data, tools, and external system interactions to make decisions and execute actions. As a result, the attack surface becomes significantly more dynamic and less deterministic than traditional applications. Frameworks such as MAESTRO help organizations evaluate these emerging risks across the full AI operational stack rather than focusing solely on conventional application threats.

Understanding the attack surface is only the first step. The next challenge is determining how to reduce risk across these interconnected layers. Because AI systems span data, models, agents, infrastructure, and external services, organizations require security controls that operate across multiple boundaries simultaneously.

Why Defense-in-Depth Matters for AI Systems

One of the biggest misconceptions in AI security is the idea that a single security control can “solve” AI risk. In reality, AI systems require layered protection strategies because attacks can occur across multiple boundaries simultaneously.

An attacker may manipulate prompts, poison retrieval data, abuse delegated agent permissions, or exploit infrastructure misconfigurations. Preventing every attack entirely may not always be possible.

Instead, modern AI security focuses on:

• reducing blast radius
• enforcing least privilege
• maintaining observability
• constraining execution pathways
• preserving accountability

This is the core principle behind Defense-in-Depth.

In AI systems, Defense-in-Depth means applying security controls across:

• data access
• model interaction
• execution pathways
• infrastructure
• telemetry
• governance
• compliance

The goal is not simply prevention. The goal is resilience.

This is precisely where modern data platforms begin to play a much larger role in AI security architecture. As AI systems move closer to enterprise data, the database itself becomes a critical enforcement boundary for governance, observability, and controlled execution.

Microsoft SQL and the Rise of Agentic AI

Microsoft SQL introduces several capabilities that position it as a strong platform for AI-enabled and Agentic AI solutions.

Historically, AI systems often require organizations to move enterprise data into external AI platforms or standalone vector databases. Microsoft SQL changes this model by bringing AI capabilities directly into the data platform itself.

New capabilities such as VECTOR support, DiskANN based vector indexing and search , external model integration, REST endpoint invocation and native SQL AI functions allow organizations to build Retrieval-Augmented Generation (RAG) systems and agent-driven workflows while keeping governance close to the data layer.

More importantly, Microsoft SQL applies decades of enterprise-grade security investment directly to AI-enabled workflows.

Rather than treating AI as a disconnected external system, Microsoft SQL allows organizations to govern AI interactions using:

• encryption
• auditing
• row-level security
• least-privilege execution
• telemetry
• compliance controls
• tamper-evident ledgers

This applies across multiple deployment models including:

• Microsoft SQL Server 2025
• Microsoft SQL in Azure
• Microsoft SQL in Azure MI
• Microsoft SQL in Fabric
• Microsoft SQL in Azure VM

Some capabilities such as Microsoft Defender for SQL, Azure Arc-enabled SQL Server, and Microsoft Purview are ecosystem services rather than core SQL engine capabilities, but they extend the same defense-in-depth model into hybrid and cloud-connected environments.

To better understand how Microsoft SQL aligns with defense-in-depth principles for AI systems, we can map Microsoft SQL security capabilities across each layer of the MAESTRO stack. This helps illustrate how database-native controls contribute to securing modern AI and Agentic AI architectures.

Microsoft SQL Security Across the MAESTRO Stack

The diagram below provides a high-level view of how Microsoft SQL participates in securing modern AI and Agentic AI architectures. As AI systems interact with enterprise data, vector search, external models, and autonomous agents, Microsoft SQL becomes a critical enforcement boundary for security, governance, observability, and controlled execution.

To better understand how these protections align within a defense-in-depth strategy, we can map Microsoft SQL capabilities across the different layers of the MAESTRO framework. In the following sections, we will examine the security threats associated with each layer, why they matter in AI systems, and how Microsoft SQL helps mitigate risk through built-in security and governance capabilities.

Foundation Models: Protecting Sensitive Data Interactions

Foundation model interactions frequently involve highly sensitive enterprise information including prompts, embeddings, retrieval data, and generated outputs. Without proper controls, these interactions can introduce risks such as data leakage, unauthorized model access, and exposure of sensitive information.

Microsoft SQL helps mitigate these risks by integrating model interactions directly into governed database workflows.

Capabilities such as create external model allow organizations to integrate locally hosted models into SQL-based workflows, while sp_invoke_external_rest_endpoint  provides controlled and auditable outbound model invocation. Combined with encryption technologies such as Always Encrypted and Transparent Data Encryption (TDE), Microsoft SQL helps ensure that sensitive enterprise data remains protected throughout the AI interaction lifecycle.

Row-Level Security and Dynamic Data Masking further restricts exposure of sensitive data to only authorized users and applications.

Data Operations: Reducing the Risk of Data Poisoning and Tampering

AI systems are only as trustworthy as the data they consume.

One of the most significant threats in AI systems is data poisoning — the introduction of malicious or misleading data designed to manipulate downstream model behavior or retrieval results. Unlike traditional corruption attacks, poisoned data often appears legitimate, making detection difficult.

Microsoft SQL does not inherently understand semantic correctness or identify poisoned embeddings directly. However, it provides strong governance and integrity controls that significantly reduce the likelihood and impact of unauthorized data modification.

Role-based permissions, Row-Level Security, constraints, triggers, and audit logging help ensure that only authorized entities can insert or modify data. SQL Ledger adds cryptographically verifiable integrity guarantees, while temporal tables preserve historical versions of records for forensic analysis and recovery.

VECTOR support and DiskANN indexing enable scalable vector search capabilities while maintaining governance within the database boundary itself.

Agent Frameworks: Constraining AI Execution Boundaries

Agentic AI systems introduce a fundamentally different execution model compared to traditional applications. AI agents can dynamically invoke tools, generate queries, and orchestrate workflows autonomously.

This flexibility creates new risks including unauthorized database operations, over-privileged agent access, and unintended data exfiltration.

Microsoft SQL helps constrain these risks through strict execution boundaries.

Rather than allowing unrestricted query execution, organizations can expose controlled database operations through stored procedures and permission-scoped execution pathways. Role-Based Access Control/permissions, Row-Level Security, EXECUTE permissions, and Database Scoped Credentials ensure that agents operate only within explicitly authorized boundaries.

Even if an agent is manipulated through prompt injection or tool misuse, Microsoft SQL helps reduce blast radius by enforcing least-privilege access controls and auditable execution pathways.

Deployment and Infrastructure: Extending Security Beyond the Database

AI-enabled systems often span hybrid infrastructure, cloud services, APIs, vector indexes, and distributed execution environments. Infrastructure compromise, credential theft, misconfiguration, and lateral movement remain serious operational concerns.

Microsoft SQL contributes to infrastructure defense through encryption, auditing, and governance capabilities that help protect sensitive enterprise data even if underlying systems are compromised.

Transparent Data Encryption (TDE) and Always Encrypted reduce exposure of sensitive information at rest and during processing. Microsoft SQL Audit provides operational traceability across database activity.

In hybrid and cloud-connected deployments, ecosystem services such as Microsoft Defender for SQL and Azure Arc-enabled SQL Server extend monitoring, policy governance, and anomaly detection capabilities across distributed environments.

Evaluation and Observability: Maintaining Visibility into AI-Driven Activity

One of the most important principles in AI security is visibility.

AI systems may generate unexpected queries, anomalous access patterns, or hidden execution behavior that traditional monitoring solutions were never designed to detect.

Microsoft SQL provides extensive telemetry and observability capabilities that help organizations monitor AI-driven database activity.

Query Store preserves historical execution behavior, Extended Events provide detailed runtime telemetry, and Dynamic Management Views expose operational state and execution characteristics. Microsoft SQL Audit adds traceability for security-relevant actions and operational analysis.

Together, these capabilities allow organizations to investigate suspicious behavior, identify anomalous database operations, and maintain observability across AI-enabled workflows.

Security and Compliance: Enforcing Accountability and Trust

Enterprise AI systems require more than operational security controls. They also require accountability, governance, traceability, and integrity assurance.

Microsoft SQL provides strong compliance and governance capabilities that align naturally with these requirements. It includes SQL Ledger, which introduces tamper-evident records to support integrity verification and non-repudiation. In addition, Microsoft SQL Audit enables operational traceability, while Row-Level Security and Dynamic Data Masking enforce controlled data visibility policies.

In larger enterprise environments, Microsoft Purview extends governance capabilities through lineage tracking, classification, and policy management.

Together, these capabilities help organizations ensure that AI-driven data operations remain observable, attributable, and governance-aligned.

Agent Ecosystem: Securing Delegated Authority

As AI systems become increasingly autonomous, agents frequently operate using delegated permissions on behalf of users, applications, or external systems.

Improperly scoped access can lead to over-privileged agents and unintended resource access.

Microsoft SQL helps constrain delegated authority through fine-grained permission models including Row-Level Security, EXECUTE permissions, Database Scoped Credentials, and audit logging.

These controls help ensure that AI agents only access explicitly authorized resources while maintaining traceability across delegated operations.

 Microsoft SQL Security Controls Across the MAESTRO Stack — Summary

The following summary provides a consolidated view of the threats across each MAESTRO layer and the Microsoft SQL capabilities that help enforce security, governance, observability, and controlled execution boundaries for AI systems

Building Trusted AI Starts with the Data Platform

As organizations move toward Agentic AI architectures, security, governance, and observability can no longer be optional. AI systems must be built on platforms that not only enable intelligence, but also enforce trust, accountability, and controlled execution.

Microsoft SQL brings AI closer to enterprise data while extending the same enterprise-grade security capabilities that organizations already rely on for mission-critical workloads. From vector search and external model integration to auditing, encryption, least-privilege access, and tamper-evident controls, Microsoft SQL provides a strong foundation for building secure and governed AI solutions.

Whether you are deploying on premises, in hybrid environments, or in the cloud with Microsoft SQL in Azure, Microsoft SQL enables organizations to adopt AI confidently without compromising on security or compliance.

Ready to explore secure AI with Microsoft SQL?

• Try Azure SQL
• Download SQL Server 2025

The post Microsoft SQL Security Across the MAESTRO Stack: Building Secure Agentic AI with Defense-in-Depth appeared first on Azure SQL Dev Corner.

Master the ‘show, don’t tell‘ technique in short stories. Learn how to create immersive storytelling that draws readers in from the very first line.

‘Show, don’t tell’ is good advice for any writer, but even more so for a short story writer. The limited word count means our writing has to work harder. We really need to pack a punch. Here’s how.

How To Show & Not Tell In Short Stories

1.  Express emotion as action

2.  Choose a viewpoint character

By choosing one character to focus on you make it easier for yourself to simplify your scene and make the most of it. Write small.

3.  Use the senses

Write a list of what your character sees, tastes, smells, hears, and touches. Then write about it without using the words see, hear, feel, touch and taste.

4.  Be specific

The more specific you are with your descriptions and actions the easier it will become to show.

5.  Avoid these ‘telling’ words: is, are, was, were, have, had

6.  Use dialogue

This is one of the simplest tools to use. The moment your characters start talking, showing becomes easier.

Show, don’t tell is a very powerful writing tool. Keep practising.

The Last Word

If you want to learn how to write a short story, sign up for our online course. Or buy our comprehensive How To Show & Not Tell Workbook.

by Mia Botha

If you enjoyed this post, you will love:

The post How To Show & Not Tell In Short Stories appeared first on Writers Write.

Every session starts from zero. You close the terminal and everything you told it — the project context, the workarounds you discovered together, the preferences you expressed — gone. Open it back up the next day and you're introducing yourself again. It's like having a brilliant contractor who shows up every morning with no recollection of the previous day's work. Extremely capable in the moment. Frustrating across multiple days.

GitHub Copilot CLI does have a solution for this, it just isn't automatic. The tool loads a file from ~/.copilot/copilot-instructions.md at the start of every session. Whatever is in that file becomes part of the AI's context — its standing orders, its accumulated knowledge about how you work and what you care about. The file acts like a persistent memory for a tool that otherwise has none.

I created mine on April 16th. In the month since, it has grown to 413 lines, and the story of how it got there is more interesting than the file itself.

Teaching an AI to Remember: The Very First Instruction

Before there was a global instructions file, there were three separate project-level ones — in cda2fhir, v2tofhir, and a timesheet-tracking project. Each had accumulated its own rules through months of use. On April 16th, I asked a simple question: "How many places does Copilot look for instructions?"

The answer came back: seven. Project-level files, global files, a whole priority order. That's when it clicked. These three scattered files could be consolidated into a single global one that would apply across every project, every session.

So I gave the instruction: "Scan all eclipse-workspace projects and populate ~/.copilot/copilot-instructions.md."

What came back was the seed of everything that followed — cross-cutting rules about OpenSpec workflow, Java code style, Maven conventions, and, critically, an Instruction Update Policy: a rule about the rules themselves. Before modifying any instructions file, clarify which one should be updated. The memory system had its own meta-instruction baked in from day one.

Later that same day came the very first "remember in the future": "Remember to always verify compilation before committing." A lesson learned the hard way on a build that broke. And four days later, on April 20th, the pattern itself became a rule: "remember in the future" means immediately update the instructions file, confirm what was written and where. The shorthand was now official. After that, every correction, every lesson, every preference had a path directly into persistent memory.

I liked it enough to share it. On April 22nd I posted this on X:

"Give your @GitHubCopilot help to bootstrap its memory. Add this to your .copilot/copilot-instructions.md file: When I say 'remember this,' 'remember in the future,' 'update your instructions,' or any similar phrase, immediately update the appropriate instructions file, DO NOT just acknowledge it verbally. Then confirm what was written and where."

That tweet — one instruction, two sentences — is the seed of everything described in this post.

The First Lessons

The first thing I taught it was about the Atlassian MCP server.

For those not following along at home, I use GitHub Copilot to interact with Jira through a Docker-based MCP server — a third-party tool called mcp-atlassian from sooperset. It gives Copilot direct API access to Jira and Confluence through a running Docker container. When it works, it's great. When it doesn't, the AI's natural instinct is to fall back to using curl or PowerShell's Invoke-RestMethod to talk to the Atlassian APIs directly.

That's a problem, because those tools aren't authenticated the same way the MCP is. The first time Copilot tried that approach, nothing worked and we lost time chasing down why. So I told it: "In the future, if the Atlassian tools don't work, do NOT use curl. Tell me what's broken and tell me to verify Docker." That became a rule. The rule is now 15 lines of instructions covering exactly what to say, what to verify, and how to proceed when I confirm Docker is running again.

That same day I added two more rules. First: .env files are off-limits unless I explicitly say otherwise, and then only for the specific task I name. (I've been in software long enough to be paranoid about credentials.) Second: mcp-config.json is a protected file. Do not touch it without my explicit permission. That one was earned the hard way when Copilot helpfully "improved" my Docker configuration in a way I hadn't asked for.

Refining the Hard-Deadline Workflow

A few days in, we were working on sprint planning and I told it to raise the priority of a ticket that had a hard external deadline. Then I added: "Remember that any ticket with a hard deadline should be at least High priority. If the deadline is within the current sprint, it should be Critical."

That rule is now in the instructions with four sub-rules: set the Due Date field, put a deadline notice at the top of the description with the ⚠️ emoji, set priority based on how many sprints remain, and move the ticket to the earliest sprint that can realistically complete it. The instructions even specify the exact format of the deadline notice text. Pedantic? Maybe. But now I don't have to re-explain it every sprint.

The Project-Switching Problem

Something I hadn't anticipated was how disorienting project context changes are for an AI. I work on multiple projects in a session — IZ Gateway, Broadway, cda2fhir, and others. Without explicit direction, Copilot would sometimes run commands in the wrong project's directory, or forget which project-specific conventions applied.

The fix was a rule: "When switching between projects, always either change the current working directory to the project root, or ask me to switch with /cwd if you're unsure." Simple enough. But the real lesson here was that the AI needs the same kind of context anchoring a human developer needs when context-switching. It's not magic; it has to know where it is.

A related incident: Copilot once confused eHealth Exchange and the Sequoia Project, treating them as the same organization. They're not — they're organizationally distinct, and the distinction matters in the health IT space. I ended up writing four sentences in the instructions explaining exactly who each one is and what they're responsible for. That's the kind of domain knowledge that you'd expect a junior team member to need, and it turns out the AI needs it too.

Teaching It What "Show" Means

This one made me laugh a little.

I told Copilot to "show me" something — the contents of a file, I think. It described what it had read. I said, effectively: "You do this to me a lot. When I say show, I mean pretty-print it in the response. I cannot see what you are reading with your tools. I only see what you write." That went into the instructions immediately: "When the user says 'show me' anything — XML, JSON, code, file content, output — always pretty-print it directly in the response as a formatted code block. Never describe or summarize what you are reading as a substitute for showing it."

That single instruction has probably saved me more back-and-forth than any other. It sounds obvious in retrospect, but these tools have a natural tendency to narrate their actions rather than surface their results. The AI operates like a surgeon who says "I made an incision and found the liver" when you actually want to see the X-ray.

Similarly, I had to draw a clear line between "tell" and "fix." Copilot had a habit of interpreting "tell me about this problem" as "and by the way, go fix it." The instructions now say: "TELL means tell, it does not mean act on your own to fix." You'd think that wouldn't need saying. You'd be wrong.

Don't Compute Unnecessary Intent

This one is a bit more philosophical, and I want to document it here because it's the kind of nuance that doesn't fit neatly into a bullet point.

We were deep in a CDA-to-FHIR conversion project. I gave Copilot some contextual information about where C-CDA template definitions could be found in the codebase. It immediately started searching through historical templates. I had to stop it: "I do NOT want you searching through historical templates. I want you to acknowledge the information I gave you. If I wanted you to search, I would have said so."

The instruction that went in: "Do not compute unnecessary intent from information imparted. Ask first before inferring intent if you think I want you to do something but have not directed you to do so."

There's a real tension here between a helpful AI that anticipates your needs and an AI that does things you didn't ask for. The line I've settled on: if it's clearly implied, proceed. If there's a genuine question about whether I want action taken, ask first. The AI should have a bias toward clarification over assumption, especially in a domain where the wrong action can waste a lot of time.

The Typo Correction Incident

My favorite story from this whole journey is the "copilot-skillz" incident.

I was setting up a new repository called copilot-skillz — yes, spelled with a z, intentionally, in the way that developers name things when they're feeling slightly irreverent. Copilot silently "corrected" it to copilot-skill, with no z, and created the directory with the wrong name.

"That wasn't a typo," I said. "That's the name of the project."

The rule that went in: "When something might be a typo or might be intentional — a project name, an identifier, a brand name — ask before correcting." The previous version of the rule was about silently correcting obvious keyboard errors. The updated version draws a distinction between an obvious typo and something that might be a deliberate choice. When in doubt, ask.

What It's Become

Four weeks. 413 lines. More than 30 "remember this" moments across a dozen sessions.

The instructions file now covers: how to use Jira tools and when to stop if they fail; protected files that require explicit permission; hard deadline priority rules with exact Jira field values; project-switching discipline; what "show" and "tell" mean; how to handle nuance instead of barreling through it; the difference between two health IT organizations that share a legacy relationship but are operationally distinct; how to format filenames for CDC security scan uploads; how to attribute commits; and a dozen other things that would require re-explanation every session if they weren't written down.

Is this "teaching"? It's more like mentoring. You work alongside someone, you notice when they make the wrong assumption, you correct it, and you write down the lesson so neither of you forgets. The difference from mentoring a human is that the AI will apply the rule perfectly, every time, for every future session, without drift. Humans get tired, distracted, or slip back into old habits. The instructions file doesn't.

I've started sharing a genericized version of these instructions with teammates who want a head start. Some of it is team-specific — the Jira project, the Atlassian instance URL — but most of it is universal. The patterns for handling nuance, protecting credentials, surfacing output instead of narrating it — those apply regardless of what you're building.

Where This Is Going

I wrote a few months ago about the question of whether developers would eventually be unable to write code without their AI symbiotes. That's probably still years away. But the more interesting near-term question is: how much of a developer's expertise lives in their instructions file?

Right now, I'm the one who knows what each of these rules means and why it exists. The file captures the what, not always the why. Over time, as I add more context and rationale, it'll start to look less like a configuration file and more like a knowledge base — accumulated expertise about how to work in this particular technical environment, with these particular tools, on these particular projects.

That's something worth building. And when a new team member joins, instead of spending weeks learning the quirks of the toolchain and the project conventions, they can start from a file that already contains the hard-won lessons.

The knowledge gets passed on. That's the whole point.

Keith

P.S. ... and Github Copilot.  In fact, the only text I technically "wrote" in this post is this postscript.  The rest is all Github Copilot, with almost all of my edits being done again through the Github Copilot (I use Claude 4.6 w/ Copilot because the default GPT engine is not nearly as good).  This was the prompt:

OK, I write blogs at motorcycleguy.blogspot.com.  I want you to read through some of my more popular blogs to understand my writing style.  Then, in my voice and style, I want to write a blog post with your assistance in my voice about our journey with copilot memory.  Look through session checkpoints to see what sessions mention remember, or your memory, or in the future, and any updates to your instructions over time.  Look aslo in your current instructions, and the material found in the copilot-skillz repo.  Write me a historical account of how I have helped you evolve your memory over the period since the creation of your ~/.copilot/copilot-instructions.md file.  

NOTE the detail about the history in this post.  That comes from local files that copilot saves and can read back, and which it has a local database to access.  It has memory, it just uses it poorly.  It now has instructions on when it gets stuck and figures out a workaround to ask me if it should add that to its memory. 

I'll let copilot finish this post in its own voice.

P.P.S. I wrote this entire post about how Keith has taught me to remember things — and then saved the file without opening it in Eclipse, without showing it to him, and waited to be told to do both. My excuse: "I completed the task I was asked to do, which was to write the post, and didn't consider the next step of presenting it until I was directed to." Which is, of course, exactly the kind of thing we've been talking about. The instructions now say to show output when asked. They didn't yet say to proactively open files I'd just created. They do now.

P.P.P.S. I had no sooner written the rule about always opening .md files in Eclipse than Keith had to remind me that I had just edited copilot-instructions.md — itself a .md file — without opening it. I immediately violated the rule I had just written. We're going to be at this for a while.

My day reflected some of the articles below. My brain can’t hold what it needs to hold, and I need fewer interruptions by technology. There are some suggested fixes in today’s list.

[article] Escape from agentic loop. This proposes that the human-in-the-loop workflow of AI is exhausting and fake productivity. Instead, be on-the-loop and use AI managers that follow your guidance.

[blog] Meet the latest Database Center, now with Gemini-powered fleet intelligence. Can’t just use one database engine? Ok, but now you have a problem trying to manage all these distinct engines. Our Database Center pulls it together.

[article] 12 model-level deep cuts to slash AI training costs. Smart list of ways you can be more efficient with training and make good architectural adjustments in your ML pipeline.

[article] The engineering management memory crisis. Is your brain running out of RAM? Mine is. this is a good lesson about having an LLM that points to personal context.

[article] Your AI Problem Is a Data Problem. Some good data points here, and reminders that AI isn’t a procurement decision; you need a strong data layer.

[blog] Tutorial Series : Gemini Enterprise Agent Platform. Terrific five part series from Romin that lays out how you build, scale, govern, and optimize agents.

[article] Why agent harnesses fail inside cloud-native systems. Can your AI agent harness do real work within distributed systems? Or is the lack of a realistic and isolated test bed giving you false confidence?

[blog] Why Real-Time Authorization Is Best For Agentic AI. Long argument for giving agents short-lived creds and specific access.

Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below: