Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
160715 stories
·
33 followers

Microsoft built a prompt injection detector. Then it caught a phishing campaign instead.

1 Share
dice

Microsoft flagged a phishing campaign last week that exploits a gap in how machines read text. Attackers are slipping invisible Unicode tag characters into email bodies; they don’t render on screen but change the underlying string that software processes.

The company says attackers are already using the technique at scale to bypass spam filters and ML-based classifiers, and the same approach could cause problems for AI systems that regularly ingest text from external sources.

Tag characters split keywords

Security researchers have documented a nearly identical technique targeting LLMs, commonly called “ASCII Smuggling,” which uses Unicode tag characters in the U+E0000 to U+E007F range — code points that exist in the character stream but aren’t displayed by most interfaces.

That gives you two versions of the same text: what a person reads and what software receives.

For example:

Human view:     funding
Under the hood: fun⟨U+E0020⟩ding

In the campaign tracked by Microsoft Defender for Office 365, attackers weren’t using tag characters to smuggle hidden instructions into an AI model. They placed them inside high-signal words associated with financial phishing, such as “funding,” “loan,” and “credit,” so that filters scanning for those terms would no longer find an exact match.

A hunting signature for ASCII Smuggling fired on roughly 21,000 messages the day before the campaign started and the next day, it fired on more than 1.3 million. Then, just two days later, the count passed 2.3 million. The whole time, recipients saw ordinary-looking offers for business loans and credit lines.

Just two days later, the count passed 2.3 million.

Tokenizers parse them differently

NLP systems break text into tokens before processing it, and slipping an unexpected Unicode character into a word can change how those tokens are formed. Researchers have already shown that encoding techniques can hide adversarial content from AI systems, although Microsoft’s campaign uses the trick for a different purpose.

NLP systems break text into tokens before processing it, and slipping an unexpected Unicode character into a word can change how those tokens are formed.

Exactly what happens depends on the tokenizer. Some may ignore the tag character while others split the surrounding text differently, so developers have to test the models they’re actually using rather than assume they’ll all behave the same way.

Running the text through standard Unicode normalization won’t necessarily remove the tags, either. NFC and NFD can clean up different representations of the same character, but they weren’t designed to strip Unicode tag characters, which means those tags can still make it through to the next step.

Agents lack email’s defenses

Email providers have other ways to spot a suspicious message beyond the words it contains, but an AI pipeline may be working with far less information.

That then becomes a problem when agents are pulling in outside text and using it to decide what to do next because those invisible characters buried in the text can change how it gets processed along the way, while also making a hidden prompt injection much harder for someone looking at the original to catch.

Normalize before the model

For applications that have no reason to accept characters in the U+E0000 to U+E007F range, the simplest approach is to remove them before the text reaches the model, although that gets trickier when an application has a legitimate reason to keep them.

In those cases, developers can compare the original text with a version that has the tags removed and look for anything that changed, while also testing the tokenizer their application actually uses to see how it handles the same characters. Whatever gets cleaned should stay that way through the rest of the pipeline, rather than checking one version of the text and then sending the untouched original to the LLM.

The subdivision flag edge case

Stripping every Unicode tag character isn’t always safe because some serve a legitimate purpose. The subdivision flag emojis for England, Scotland and Wales rely on invisible tag-character sequences to render, and Microsoft’s initial hunting signature was broad enough to trip on those flags before the team carved out an explicit exception.

Stripping every Unicode tag character isn’t always safe because some serve a legitimate purpose.

The post Microsoft built a prompt injection detector. Then it caught a phishing campaign instead. appeared first on The New Stack.

Read the whole story
alvinashcraft
2 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Audacity 4 is a complete revamp of the ‘world’s most popular’ audio editor

1 Share
Audacity 4 running on a laptop.

Audacity 4 has been in the works for some time, and had its own mini controversy last year when an unfortunate redesigned logo started making the rounds. The final version of the new icon isn't nearly as bad as the early rendition that was circulating last October. But more importantly, all the promised improvements to the self-proclaimed "world's most popular audio editing and recording app" are here, turning Audacity from a free and capable but extremely clunky audio editor into a powerful modern tool.

The biggest change is the move to a non-destructive editing workflow. That means you can trim a clip down, then recover the audio simply …

Read the full story at The Verge.

Read the whole story
alvinashcraft
2 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Agentic Loops for Knowledge Workers

1 Share
From: AIDailyBrief
Duration: 53:11
Views: 2,382

Agentic loops let AI run continuously against a measurable goal instead of waiting on prompt-by-prompt direction, and the technique is finally moving beyond software engineering. NLW and Nufar Gaspar walk through how to structure knowledge work tasks so agents can self-check and keep going until the job is actually done. This full workshop session covers loop design, graph engineering, and getting agents to operate autonomously as a group.

The AI Daily Brief helps you understand the most important news and discussions in AI.
Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614
Get it ad free at http://patreon.com/aidailybrief
Learn more about the show https://aidailybrief.ai/

Read the whole story
alvinashcraft
3 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Four Years of Writing Every Week

1 Share

Four years and 210 weekly issues taught me more than the topics did. The four lessons I keep coming back to are the ones this issue walks through: consistency beats brilliance, teaching is the fastest way to learn, the fundamentals outlast the frameworks, and your audience is smarter than you think. I also want to talk about AI, because the next four years of this job will not look like the last four.

Four years ago this week, the first issue of this newsletter went out.

Two hundred and ten Saturdays later, something has landed in your inbox almost every single week. Whether I was traveling, sick, buried in work, or completely out of ideas.

I want to do something different this week. Instead of an EF Core trick or an architecture deep dive, I want to step back and talk about what writing four years of these actually taught me. The lessons turned out to be bigger than the topics.

Timeline of four years of the newsletter, from issue 1 in September 2022 with a few hundred readers to issue 210 in September 2026 with over 66,000 readers, with the releases of .NET 7 through .NET 10 marked along the way and .NET 11 just past the end

I Almost Didn't Make It Past Issue 10

The first issues were rough.

I wasn't sure anyone wanted to read what I had to say. I'd write something, second-guess it, rewrite it, and hit send with my stomach in a knot. A few people replied, most didn't, and there were weeks I was convinced I'd run out of things to write about by issue 20.

What kept me going was a decision I'd already made: one issue, every week, no exceptions. Motivation alone would never have done it, because motivation shows up when you don't need it and vanishes when you do.

That decision did more for my career than any single piece of code I've ever written. I wrote about that same itch to build in The Urge to Build Something, and this newsletter became the longest-running thing I've ever built.

Lesson 1: Consistency Beats Brilliance

Not one of these 210 issues was perfect. Some I'm proud of, some I'd rewrite completely, and a few I'd quietly delete.

But the brilliant issue you never publish helps no one. The decent issue you ship every single week, for four years, compounds into something neither of us could have predicted at the start.

It's the same lesson I keep relearning in software. The clever architecture that ships to production in eighteen months loses to the boring one that ships to production every two weeks.

If you're sitting on a blog post, a side project, or a talk you haven't given, this is the only advice that matters: start, and then don't stop. Quality grows out of quantity.

Lesson 2: Teaching Is the Fastest Way to Learn

I thought I understood things, until I tried to explain them to thousands of developers who would absolutely email me when I got it wrong.

You can hold a fuzzy, half-correct model of a concept in your head for years and never notice the gaps. The moment you sit down to write 1,500 clear words about it, every gap suddenly becomes obvious. Why does this pattern exist? What problem does it actually solve? When would I not use it?

A huge amount of what I know about .NET, architecture, and system design, I learned because I had to explain it on a deadline. Writing the newsletter forced me, week after week, to actually know the things I thought I already understood.

If you want to understand something deeply, teach it. Write the post, record the video, or explain it to a junior on your team. Writing the explanation is where the learning happens.

Lesson 3: The Fundamentals Outlast the Frameworks

Look back across four years and you'll see the surface change constantly. .NET 6 became 7, then 8, then 9, then 10, and .NET 11 is right around the corner. Libraries came, went commercial, and got replaced. Patterns I championed, I later qualified.

But the advice I keep giving hasn't changed since issue 1:

  • Start with a modular monolith. You can extract a service later, but you can't easily un-distribute a system. Split when something concrete forces you to (a team blocked on every deploy, a module with its own scaling needs), not because the monolith feels messy. I believe this enough that I built an entire course around it.
  • Keep business rules inside the domain model. A rule that lives in a controller or a request handler gets duplicated the day a second caller needs it, and the copies drift apart. The model enforces it once.
  • Treat your API as a long-term contract. Consistent error responses, pagination, and versioning cost you an afternoon on day one. Retrofitting them after clients depend on the API costs you months. That thinking became Pragmatic REST APIs.
  • Assume every network call will fail. Which means retries, and retries mean idempotency and the Outbox pattern for anything that publishes messages.

That list was true in issue 1, and it'll be true in issue 420.

Spend most of your learning budget on the things that don't expire. The new syntax is the easy part once the underlying ideas are solid.

Lesson 4: Your Audience Is Smarter Than You Think

Some of the best thinking in these four years didn't come from me. It came from your replies: the corrections, the "have you considered…" emails, the pushback when I oversimplified, the war stories from production systems far bigger than anything I've run. Every time I wrote something a little too confidently, someone wrote back with the nuance I'd missed, and the next issue was better for it.

Writing in public is the best feedback loop I've ever found. It's terrifying to be wrong in front of an audience, and it's the fastest way to get less wrong.

Thank you for being demanding readers. You made me a better engineer.

What This Newsletter Actually Became

I started writing to share a few .NET tips. Somewhere along the way it became the spine of everything else: the videos, the courses, the community, this whole second career I never planned. The YouTube channel turned four this year too, and I told that side of the story in What 4 Years on Tech YouTube Taught Me.

But strip all that away and the core is still just this: every week, I try to explain one thing clearly to people who want to get better at their craft. That's it. That's the whole job, and it's the most rewarding work I've ever done.

When someone emails to say a piece helped them pass an interview, ship a system, or finally understand something they'd struggled with for years, that never gets old. That's the reason there's a 210th issue at all.

What About AI?

I can't write a four-year retrospective in 2026 and not talk about AI.

When issue 1 went out, AI meant a slightly smarter autocomplete. Today, coding agents write a growing share of new code, and the argument has moved on to whether a human should even read the output. Uncle Bob, the author of Clean Code, says he stopped reading the code his agents write. I recorded my reaction to that one, because I think it's the most interesting question in software right now.

I'm very bullish on AI. I've spent a good part of this year going deep on agentic coding, and I shared my whole setup in a video I called Tokenmaxxing: My Claude Code Workflow.

Notice what AI didn't change on the list of lessons above.

The fundamentals became the review checklist. Whether a human or an agent wrote the code, I ask the same questions: does this rule belong in the domain model, will this change break an existing API client, what happens when this call fails. The agent changes how fast the code appears, not what makes it correct.

And teaching turned out to be the perfect preparation. Directing an agent well is mostly the skill this newsletter forced me to practice for four years: explaining a problem clearly enough that someone else can act on it.

Expect more AI in these issues going forward.

Thank You

I don't say this enough, so let me say it plainly.

Thank you for reading. Thank you for the four years. Thank you for forwarding issues to teammates, for replying with your own hard-won lessons, for trusting me with a few minutes of your Saturday. Some of you have been here since the early, rough issues. Some of you joined last Tuesday. Either way, you're the reason this exists.

I have no plans to stop. There are at least 200 more issues of ideas rattling around, and the urge to build, to explain, and to share hasn't faded one bit.

So here's to the next four years. Same deal as always: one issue, every week, no exceptions.

If you've gotten value from these over the years and want to go deeper into the way I actually build software, that's exactly what I put into Pragmatic Clean Architecture.

See you next Saturday.

Thanks for reading.

And stay awesome!




Read the whole story
alvinashcraft
3 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

v1.0.13

1 Share

Feature: cancellation for host-owned external tools

Host-owned external tool callbacks are now cancelled when their runtime request completes or their SDK session terminates. The cancellation primitive is idiomatic per SDK: .NET passes a request token to AIFunction, Node.js exposes ToolInvocation.signal, Go cancels ToolInvocation.TraceContext, Java cancels the returned CompletableFuture, Python cancels the handler task, and Rust drops the handler future. Go handlers that retain TraceContext for background work must derive a separate lifetime because the invocation context is cancelled when the request ends.

Feature: declare application identity with client info

Client options now accept optional client info (application name and version, integration name and version) across all six SDKs, exposed idiomatically per language (clientInfo in Node.js, client_info in Python and Rust, ClientInfo in Go and .NET, setClientInfo in Java). When set, the SDK forwards it on the server.connect handshake so the telemetry the runtime emits on the connection is attributed to the application and its Copilot integration instead of the runtime's own build. All fields are optional, and leaving client info unset keeps the runtime's default attribution. See Client info.

Feature: Node Agent Factories pagination and run notifications

The experimental Node.js Agent Factories convenience API now supports paginated run history. Existing session.factory.listRuns() calls still return the runs array, while calls with afterSeq, beforeSeq, or limit return the full page with cursor and truncation metadata.

Factory run and resume options now accept notifyOnComplete and logPhaseNames. The SDK forwards these options to the Copilot CLI for new and resumed runs.

Feature: selectable ask_user session behavior

Session create and cold resume now accept a language-specific askUserVariant option with legacy and elicitation values. SDK sessions retain the legacy question-and-answer tool by default. Select elicitation and provide an elicitation handler to expose the structured form-based ask_user tool.

Feature: rotating session-scoped GitHub credentials

All six SDKs can now acquire short-lived GitHub credentials through a session-scoped callback. The SDK registers the callback before session create or resume, maps initial and refresh requests to the owning session, and removes registrations on rollback, replacement, session close, and client close. Static per-session gitHubToken credentials remain supported and are mutually exclusive with the callback.

Token responses use the shared tagged token/cancelled shape and require expiresIn, expressed as the positive number of seconds remaining when the callback completes. See github/copilot-agent-runtime#16381 for the runtime credential-authority implementation.

Initial acquisition occurs during create or resume; cancellation, callback errors, and invalid credentials reject that operation instead of falling back to ambient authentication. Idle sessions refresh only before their next credential-consuming operation.

Feature: extensions can request sensitive environment variables

Copilot CLI extensions can now ask for named sensitive environment variables when they join a session. joinSession() accepts a requestedEnvironmentVariables option listing the variable names the extension needs. The CLI shows a permission prompt naming the extension and the exact variables requested. On approval, only those variables reach that extension and their values are written into the extension process's process.env before joinSession() resolves. On denial, joinSession() rejects, the extension does not load, and its tools never reach the model.

An approval is remembered against the exact set of names the user saw, so an extension that later asks for one more variable prompts again. Names that are unset, or that the CLI does not filter from extensions, are not prompted for. This is the client half of the feature; it requires a Copilot CLI that supports extension environment access, and older CLIs ignore the request and grant nothing.

import { joinSession } from "@github/copilot-sdk/extension";

const session = await joinSession({
    requestedEnvironmentVariables: ["GITHUB_TOKEN"],
});
const token = process.env.GITHUB_TOKEN;

Feature: early session-event subscription (Rust)

The Rust SDK can now observe every event routed to a session, starting with that session's very first routed event. Client::prepare_session and Client::prepare_resume_session return an inert PreparedSession that owns the session's event channel, so a subscription can be installed before any protocol activity begins:

let prepared = client.prepare_session(
    SessionConfig::default().with_event_buffer_capacity(2048),
)?;
let mut events = prepared.subscribe();
let session = prepared.start().await?;

Previously, Session::subscribe could only be called on the returned session, so events the runtime emitted while session.create / session.resume was still in flight were broadcast with no receiver installed and dropped. Ephemeral events such as session.idle are not persisted, so they could not be recovered with getMessages either.

The guarantee is scoped to routed events. For cloud sessions where the server assigns the session ID, the SDK cannot route notifications until the session.create response arrives and the ID is known, so events emitted before that point are not routable to any session. Pin session_id on the config to get router registration before the RPC, and with it complete pre-response coverage.

prepare_* is synchronous and inert: it validates the buffer capacity and allocates a local channel, and performs no router registration, task spawn, or wire activity until start() is first polled. start(self) consumes the handle and PreparedSession is not Clone, so a prepared session can never produce two event loops. Dropping an unstarted handle leaves no state behind; dropping a polled start() future cancels the startup and unregisters the session, so a retry with the same session ID succeeds. Session registrations now carry an ownership identity, so cleanup removes only the exact registration it owns and an abandoned startup can never evict a same-ID retry (or a session that replaced it).

Both SessionConfig and ResumeSessionConfig gained a runtime-only event_buffer_capacity option (default 512, Some(0) rejected as an invalid config). The buffer is finite, so slow subscribers observe Lagged rather than applying backpressure; consumers that need a lossless view of a large startup burst must size the buffer accordingly or drain concurrently with start().

create_session and resume_session are unchanged wrappers over prepare_*(...)?.start() with identical RPC sequences and error kinds.

Feature: host-injected managed settings permissions

Session create and resume accept a new optional managedSettings option that injects an enterprise permissions policy at session startup, alongside the existing enableManagedSettings self-fetch flag. The current contract is permissions-only: disableBypassPermissionsMode (the literal "disable"), plus deny, ask, and allow rule lists. The layer composes restrictively with any server- or device-level managed settings (deny/ask are unioned, every present allow list must admit a tool, and disableBypassPermissionsMode is deny-wins).

This layer is startup-only and is not persisted with the session, so it must be re-supplied on resume to remain in effect; omitting it on resume clears the previously injected layer. It can be combined with enableManagedSettings. Host injection requires Copilot CLI 1.0.79-5 or later and does not require an SDK protocol version bump.

The generated session-event types also expose truthful injected-policy provenance: session.managed_settings_resolved can report source as client or mixed, with optional clientManaged metadata.

const session = await client.createSession({
    managedSettings: {
        permissions: {
            disableBypassPermissionsMode: "disable",
            deny: ["shell(rm*)"],
            ask: ["write"],
        },
    },
});
var session = await client.CreateSessionAsync(new SessionConfig
{
    ManagedSettings = new ManagedSettings
    {
        Permissions = new ManagedSettingsPermissions
        {
            DisableBypassPermissionsMode = DisableBypassPermissionsMode.Disable,
            Deny = ["shell(rm*)"],
            Ask = ["write"],
        },
    },
});

Feature: Auto model routing tier controls

Sessions can now steer auto model routing toward efficiency, balance, or intelligence. An Auto tier can be set at session creation, and a new setAutoTier (and equivalent setModel option) lets sessions stage or reset a tier preference afterward, since the runtime only commits a staged preference on the next successful auto model turn. (#2437, #2514)

await session.set_auto_tier("efficiency")

Feature: sandbox bypass and non-object external tool arguments

Sandbox configuration now exposes allowBypass across all six SDKs. External tool overrides such as apply_patch can also receive non-object JSON argument values, which previously failed before reaching the host handler in .NET. (#2372, #2496)

Feature: host-resolved feature flag overrides

Session create and resume now accept a featureFlags map across all six SDKs, forwarding host-resolved overrides while preserving the distinction between an unset map and an explicitly empty one. (#2451)

Other changes

  • feature: use session.detach instead of session.destroy for SDK session cleanup so disconnecting one client no longer tears down a shared session for other owners (#2307)
  • bugfix: [Rust] answer the request ID when a tool handler panics (#2311)
  • bugfix: [Go] close failed session event loops (#2360)
  • bugfix: support bracketed IPv6 runtime URLs (#2200)
  • bugfix: [Python] serialize native values in tool results (#2374)
  • bugfix: [Rust] prevent orphaned CLI processes (#2292)
  • improvement: [Rust] default ClientMode::Empty to no built-in skills (#2410)
  • improvement: [Go] auto-detect bundler package name and avoid duplicate license downloads (#2452, #2453)

New contributors

  • @lukehoban made their first contribution in #2292
  • @scordio made their first contribution in #2382
  • @OllieinCanada made their first contribution in #2374
  • @gimenete made their first contribution in #2458
  • @gwwar made their first contribution in #2464
  • @Pybsama made their first contribution in #2163
  • @green3sf made their first contribution in #2360
  • @gokhanarkan made their first contribution in #2532

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by Release Changelog Generator · copilot · auto · 125.2 AIC · ⌖ 8.09 AIC · ⊞ 11.7K

Read the whole story
alvinashcraft
3 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

From AI Code to Trusted Software: Harness Engineering in Practice

1 Share

Your team is shipping more AI-written code every quarter. How do you know it is any good?

Good means it meets your standards, and you have probably already tried handing your agents the standards: a context file, a style guide, the wiki pasted into the prompt. The agent reads them, agrees, and still breaks them, because instructions to a model are suggestions. What the repository permits is what actually happens. Your people absorb standards through review comments and hallway corrections, and the lessons stick. An agent apologizes and forgets by the next session. The only place its lessons can accumulate is the repository itself.

That gap used to be an annoyance. With AI doing real engineering work, the quality gap is the whole game.

Harness engineering is the discipline that closes it. It is the process of imbuing a repository with your standards so that the repository itself enforces them: permissions and boundaries that say what any actor may touch, quality gates that fail closed, evidence attached to every change, and observability that spans runs rather than moments.

Models supply software delivery capacity. The harness supplies observable accountability: every change carries what was done, which rule allowed it, and what happened as a result, no matter which model, agent, or person did the work.

Why now

The industry started using the term harness engineering back in February, 2026. Since then, Thoughtworks, LangChain, and others have built serious thought leadership around the same shape. When several firms independently converge on the same word, it usually means they are trying to name the same problem.

The real problem is this: AI pilots demo brilliantly and then stall on the way to production. Not because code generation is weak; that stopped being the bottleneck some time ago. The bottleneck is verification.

An agent can produce a plausible change faster than your team can decide whether to trust it, and a backlog of plausible-but-unverified changes is not velocity. It is inventory and technical debt waiting to break something in the middle of the night.

This is not just the stuff of software wonks and conference talks. Engineers are publicly building personal software factories: agent loops that pick up issues, write code, validate it, and report back overnight. Enterprises like Ramp, Stripe, and DoorDash have built their own internal versions to drive business outcomes and efficiency, and vendors now sell the platform outright off-the-shelf. Some of your engineers are probably running one right now in production.

Building the harness the factory will run on is becoming table stakes to delivering agentic workflows at enterprise scale. You can buy the model access. You cannot buy the harness, because the harness is your own standards made enforceable in your own repositories. That is where the trust comes from, and no vendor ships it in the box.

There is a maturity arc in how engineering teams hand work to AI:

  • Assisted. The model suggests. A person asks, reads the code, and decides what ships.
  • Harnessed. The system executes multi-step work under gates. A person reviews the evidence and approves the outcome.
  • Risk-tiered. The gates grade the risk. Low-risk changes merge on their own, and a person reviews intent only where the risk says so.
  • Autonomous. Work enters, and the harness executes and validates it. Humans govern the boundaries, inspect the evidence, and intervene when the system falls outside them.

Figure 1. Every wider loop keeps a human checkpoint. The checkpoint moves earlier: from the diff, to the evidence, to intent on the risky cases only, to the boundaries.

Notice what stays consistent: the human is never removed from the broader workflow. The human role moves upward, and it becomes conditional before it leaves any one loop for a wider one. At the same time, more of the review, challenge, and reflection can itself become governed machine work.

Harness engineering is what makes each step toward greater autonomy safe to take.

What it looks like when it is real

Three patterns show up in every serious implementation I have seen or built on client engagements. None of these patterns require exotic machinery: everything is built from parts your repository host already has. A required check that blocks a merge is a gate, and every repository host already has one. The parts do not change as a team matures. What changes is how many of them are required.

Gates that fail closed. At its simplest, a gate is simply a required check: a change that cannot produce its evidence does not merge. That holds for a senior engineer having a confident day, and it holds for an agent that produced something plausible at 3 a.m. As the harness matures, even the gate reader stops being trusted: my factory's merge step asks the host which required checks passed, and an error in that probe counts as a failure.

When the infrastructure itself has a problem, the answer is stop, not shrug: the same system, hitting a spend limit mid-run, stops the day's work rather than degrade quietly, and a budget-stopped day cannot restart itself through retries. Silent degradation is how trust dies and how budgets are quickly exceeded.

Evidence as the working currency. In a harnessed repository, every merged change adds its own verdict to the ledger: what was checked, by what, and with what result. A mature harness adds provenance: my factory appends a reproducibility manifest to the pull request, the models, prompts, and configuration in effect at authoring time, so that months later when a root cause analysis asks "what produced this change?", the record answers on its own.

The review process stops meaning reading every diff and starts meaning auditing verdicts and sampling deeply where the evidence looks thin on the surface. That is not a lowering of the quality bar or sweeping reviews under the rug. It is the same quality bar, held at a higher altitude where humans operate day-to-day.

One caution that the measurements insist on: if the same reasoning path writes the change and defines the proof, you may only have one opinion wearing two hats. Verification needs a reviewer whose charge, evidence, or objective is independent of the author's. Model diversity is one more lever. A judge running on a different model family does not share the author model's blind spots, so the two are less likely to fail in the same way.

The old disciplines, compiled. The practices your best teams spent twenty years internalizing move from being embedded culture to configuration templates. Every project leads with tests first, small diffs, executable acceptance criteria and traceable changes. In my factory, 'test before commit' is written law, a hook blocks any agent from claiming done while a test fails, and branch protection is a versioned ruleset, not a static and outdated wiki page at the repo level.

Where this goes: the repo becomes a mini-organization

Here is the part that surprised me, and the reason harness engineering deserves an executive's attention, not just an architect's.

It did not show up in the first weeks. It showed up months in, once I understood the operator's seat: the person who runs the factory rather than working inside it. Early on I was directing the code, one instruction and one diff at a time. Somewhere along the way I began directing the system instead. I set the intent, the boundaries, and the conditions for acceptable work. The factory handled the execution and left evidence behind. What that evidence taught me changed what I asked for next. Over time, the factory began organizing those expectations into its own structure, separating how work should be done from what the resulting software had to be. I had not designed that structure in advance. It emerged before I even had a name for it.

There is an old law in this business of software engineering: software ends up shaped like the organization that builds it. A harnessed code repository runs Conway's Law in reverse. Wire enough standards, evidence, and judgment into a repository, and an organization starts taking shape inside the software.

Once your standards are executable, the repository can hold agent-based roles filling specific seats, not just generic scripts. And those seats can cover diverse responsibilities and working styles: a product seat that grooms intent into well-formed work, a quality review seat that judges changes with named charges of failure. For instance: one attacks feasibility, one attacks security, one attacks the user's experience. These are fully auditable decision records that nobody, human or machine, can skip. Once implemented, immutable ledgers can track whether each reviewer's findings survive scrutiny, so the judges themselves get calibrated over time based on your risk tolerance and working style.

These role-based seats convert to autonomous agents only when their output can be verified mechanically; where complex judgment is the verification, the seat remains a human one.

This is not speculation. In our research system, HydraFlow, repositories now convene review councils before designs land, file dissent by name, and keep calibration records on their own judges.

Humans did not disappear in this modern world of agentic software development. Their role moved up: setting and ratifying direction, defining authority, and auditing the institution, while the system increasingly participates in those same processes under articles that govern how change becomes legitimate.

A production-grade repository run this way stops resembling a codebase with bolt-on automation tooling and starts resembling something else entirely.

A framework for governing it

I wrote recently on my own blog about what six months of running a personal software factory taught me, What Matters After the Software Factory Works. The lessons eventually compressed to four words:

Purpose is what the work is for. Articles define what must remain true. Actors do the work under delegated authority. Artifacts preserve what the organization produces and remembers.

PAAA, for short, describes a simple and repeatable framework for governance.

Every established engineering organization has some version of these oversight structures. As engineering systems become more autonomous, the same structures increasingly need to exist inside the repository.

The harness makes the Articles executable. People and agents staff the Actors. Evidence, decision records, ledgers, and institutional memory become the Artifacts. Intent supplies the Purpose.

Figure 2. The four words at weekend scale. The mission writes the rules, the rules bound the builder, the work leaves evidence, and failures flow back and rewrite the rules. One closed loop, even in a test project.

This is the larger cultural implication of harness engineering. Once a repository contains not just workflows, but authority, boundaries, evidence, memory, and mechanisms for changing its own rules, it begins to resemble the organization operating through it.

Humans do not disappear from that harness engineering-guided organization. They define its purpose, grant authority, ratify its deepest rules, and audit whether the system continues to behave as intended.

What you are building when you invest in a harness is not a smarter pipeline. It is a development organization in a box, and it reports to you.

Where to start

The four framework words double as an audit your leadership team can run this quarter.

These four questions are the agenda of our half-day harness engineering working session. We run it with your engineering leaders, in your context and against your own repositories, and you leave the same day with a written answer to each question and a path forward.

Ask yourself the following questions:

  • Who sets the direction for your AI initiatives, and what happens when that direction goes silent?
  • What must always remain true across your repositories and workflows, and what enforces those rules when nobody is looking?
  • Who, or what, holds each decision role in your review process, and how would you know if a role went rogue?
  • What do your systems remember, and where does that memory live?
     

Let's workshop this out.

Read the whole story
alvinashcraft
3 minutes ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories