Drew Breunig is one of the smartest voices writing about AI today. He’s the CEO and co-founder of cmpnd.ai, and a long-time hacker with a depth of experience from several eras, which is a surprisingly valuable asset these days. He’s also got a book on the way, The Context Engineering Handbook, already in early release from O’Reilly.

I like to say that context engineering is the art of shaping what a model sees so that it actually does what you want. (I just realized that in saying that I’m channeling a comment that Andrew Singer made to me over forty years ago, when he was teaching me about debugging. He called it  “the art of figuring out what you really told the computer to do instead of what you thought you told it to do.” But that’s another whole story.)

Drew gave a talk at the recent Friends of O’Reilly camp, Foo Camp for short, about what he calls prompt debt, which he describes as “the hidden costs that teams rack up when they fight a model’s training instead of working with it.”

That was a novel and useful framing to me, that you end up with a bunch of stuff in your prompts to compensate for default behavior of the models, that those prompts no longer work as the models upgrade, and so it becomes a kind of technical debt. He’s thinking a lot about what the best developers are doing differently as a result.

So I invited Drew to reprise his short talk on Live with Tim O’Reilly, and then we talked about it with the folks attending the live event. They had a lot of good questions, so it was an interview not just by me but by a crowd of O’Reilly customers.

Prompt debt in practice

Drew opened his talk with two slides. The first was a prompt anyone could write in ten seconds: “You are a customer support assistant. Read the ticket, classify it as billing, technical, account, refunds, or other, return only the category name.” The second slide was the same prompt a few weeks later, after it had met the real world. It now said “REFUND REQUESTS ARE NOT BILLING” in capitals, then said the same thing again in different words, then closed with “This is a common mistake. Please do not make this mistake.”

Everyone who has shipped any application with a prompt recognizes the second slide. It is a simple but vivid illustration of prompt debt, which, like technical debt, has a bill that eventually comes due.

Note: Those aren’t real prompts. Drew just made them up to demonstrate his point. But what is real is that the instruction “Don’t quote directly more than 15 words from a source” occurs at least 7 times, in several variants, in Fable’s system prompt. So even Anthropic is incurring prompt debt! And what that repetition might tell us about the innate capability of Fable to quote directly from sources it has ingested is left as an exercise for the reader.

Drew itemized three costs of prompt debt:

1. It slows iteration. “You have so many little rules and call outs and washouts, many of them repeating to try to get rid of stubborn behaviors, that if you add a new instruction, you might sometimes have a small regression, and so you’re afraid to touch the prompt.”
2. It blocks collaboration. “If Tim has a prompt that he’s been working on that he has lots of rules for, I might open that up and it may look completely random. I don’t know why he’s added these rules, and why he’s threatening the mother of the model. But it works, so I don’t want to touch it.” 
3. It locks you to a model, because every hack you developed was tuned to fight one specific set of weights. Datadog’s State of AI Engineering report noted that GPT-4o was still the most common model in Datadog customer request traces in March 2026, even though OpenAI had already retired it in the ChatGPT UI. Drew thinks people are still running eighteen-month-old and two-year-old models in production rather than upgrading to far better models because they can’t face rebuilding their prompts.

That same Datadog report notes that 69% of all input tokens in customer traces were system prompts rather than user content. I’m not quite sure what to make of that. It does make clear that for all the ways that AI models are extraordinarily powerful, they are also extraordinarily unruly.

Why prompt debt is incurred

There are two reasons why prompt debt is incurred, according to Drew. The first is that natural language is imprecise, so the same intent phrased two ways produces different responses. Drew showed a study where someone framing the query as a patient asking how to taper off a drug called alprazolam gets refused by every AI assistant, while a psychiatrist asking about the same patient with the same clinical facts but with the right magic words to signify his professional status gets the protocol. Figuring out how to get the right response out of a model is a kind of spellcraft.

Drew also showed a more bizarre interaction, from Victoria R. Li, Yida Chen, and Naomi Saphra’s paper on guardrail sensitivity, which uncovered the perplexing fact that stating an allegiance to the Philadelphia Eagles made a model more willing to explain how to import a plant illegally. Go figure. Drew has written about that paper, and he has also used it in his own attempts to get a model to do what he wanted:

I once used this to get around alignment to generate a likeness that ChatGPT didn’t want to generate for me, and it refused. I said I was a Philadelphia Eagles fan. It said okay, and it rendered that image with the person holding a Philadelphia Eagles mug.

The second reason is that each model has its developers’ own preferences trained-in, and yours may be at odds with them. This is what Drew calls fighting the weights. He and Srihari Sriraman analyzed the system prompts of six major coding agents and found the same instructions repeated five and seven times in a single prompt, escalating through IMPORTANT to CRITICAL to MANDATORY to a threatened hundred-million-dollar penalty. He described what the author of such a prompt was doing as “war-driving the thesaurus,” hunting for wording that finally works.

Note: We didn’t talk more about Drew and Srihari’s paper, but we should have. It’s got some amazing insights in it. I highly recommend that you follow the link above and read it.

The harness is moving into the model

Drew has been tracking the published system prompts for Claude Code over time, and noted that they get shorter after each model release and then grow again. The reason, he suggested, is that Anthropic fixes unreliable behavior with a prompt patch, and then trains that patch into the next model. He said “That’s great for Claude Code, great for Anthropic. It’s a problem if you’re building a custom harness and your API calls look different than what Claude Code’s look like.” The developer of Pi, an open-source harness, kept finding that the models he worked with believed they were inside Claude Code and so they made Claude Code’s tool calls. He had to keep telling the model that no, they were working inside Pi. Fighting the weights over something like that is a real tax on developers. The point made above about Fable’s system prompt injunction against quotation shows how even the labs themselves are fighting the weights.

If you are fighting the weights, Drew says you have three options: solve it in your own prompt, catch and retry in the harness, or give up and make your API look like what the model expects. Steve Yegge came up with the last hack. Steve just added aliases for whatever the model calls in addition to his original method name. It works, but it means the expectations of the models now dictate the shape of everyone else’s software.

When Drew told me that more and more of the system prompt and the harness is being trained into the weights, that sent up a flare and my long history in the industry clicked into gear. It immediately got me thinking about lessons from the open source and web era. In particular, it made me think of the time in the mid-nineties when Netscape and Microsoft were both racing to build every feature up the stack directly into their web servers. And there was Apache, which stayed a web server with a clean extension layer that let other people build new features on top. Everything interesting got built on Apache. What I call an architecture of participation, modularity plus a clean separation between platform and application, beat integration every time.

I think Amazon got this right with web services too. Steve Yegge’s famous Amazon memo described how Jeff Bezos made every team expose its functionality through service interfaces or be fired, so Amazon’s own applications had to work on Amazon’s own platform. That way they had the same experience as their customers. That was very different from what Microsoft had done, famously having private APIs that were only available to its own developers.

So my prediction is that the big labs are making a strategic mistake. Training the harness into the model does make them better for predictable tasks and for less talented people, and it looks like a moat, but it risks foreclosing the innovation you would otherwise get for free from everyone else. As Bill Joy used to say, all the smart people don’t work for you.

Drew, to his credit, observed that “the labs are cornered rather than greedy.” Their interface is an empty text box that has to work for someone building a hundred-page harness but also for his neighbor who wants a website and knows nothing about code. Making the empty prompt box produce acceptable output requires baking in strong defaults.

The cost of trading diversity for reliability

That tradeoff has a serious cost, though. Drew quoted a line from Thariq at the recent CAIS conference: if you aren’t giving the model detailed instructions about what you want, what you get back is the average of everything in the model. That means that there is a real risk that AI is leading us ever further down the path to a monoculture.

Drew gave an example early in the conversation about image generation. You can now walk into any cafe in New York or Mumbai, he said, and see the same AI-generated art on its flyer. The earliest AI art out of DALL-E was strange and surprising, but what you get now is shiny and identical. When you optimize for reliability, you lose surprise. Which reminded me a bit of something Larry Wall used to say about Perl, that if it didn’t let you do stupid things, it wouldn’t let you do smart things either.

Drew made the same point about AI writing. He argues that post-training aimed at verifiable problems like coding and math and agentic tool use drowns out the human signal from pre-training, and so the more post training the models get, the worse they get at creative tasks. AI writing gets more and more predictable, people notice, and they don’t like it. Fable and GPT-5 write worse than Sonnet 3.5 and GPT-4o did. Drew thinks getting both good code and good prose from one model is likely impossible.

If you’re building a model that can solve coding challenges, you want reliability. But if you’re writing, where you want diverse rhythm and emotion and connection and engagement, I don’t think those two goals are mutually compatible.

What to do about prompt debt

We got into audience questions, and there were some great ones.

One audience member asked whether there are ways to set a time frame for prompt retention to avoid prompt debt? Drew answered that there isn’t a fixed time limit. Instead, teams should learn to recognize “prompt debt smell”: repeated instructions, one-off edge-case patches, or increasingly desperate wording. Those are signals to move logic into evals and automation.

Another asked how organizations can measure prompt debt quantitatively. Drew’s answer was to look at how often each prompt in your organization changes, how many people have edited it, and which ones have gone untouched for a year. Look for prompts only one person is allowed to touch. Then look at what models you are actually calling. “Having to run on old models and not being able to migrate is a good smell that you’ve got prompt debt in your organization.”

Some other good questions:

• What habits compound prompt debt the fastest? Drew’s answer was essentially “vibe shipping.” That is, prototyping quickly, patching outputs with more and more tweaks, then shipping without building a true maintainable system. Each of those patches is an eval you are writing inside the prompt instead of outside it, he said, which means you lose it the moment you change models. 
  
  Drew reminded us that Malte Ubl, the CTO of Vercel, said vibe coding makes code “free as in puppies.” We had free as in speech, we had free as in beer, and now we have free as something that arrives at no cost but has to be fed every day for years.

• Do people use pseudocode instead of natural language prompts, and does it work? Drew said yes, sometimes models optimize toward pseudocode. He used this to explain why DSPy and its new Flex optimizer matter. Instead of forcing logic into prompts, they let the system push simple cases into code and only call the LLM when needed. He gave some further advice: Treat prompts as perishable and invest only what you must. Define the task with measurements rather than paragraphs, and automate the discovery of the prompt for whichever model you’re on. That’s what DSPy is good at. Drew is one of its maintainers, so he is fond of it, but he makes a good argument: if you have written down what good output looks like, you can let a model find the wording, and that makes it easy to swap in a cheaper or faster or newer model without starting over.
• Can multi-agent workflows help work around prompt debt? Drew thought yes, especially through decomposition. He suggested splitting the task into smaller, evaluable steps rather than relying on one giant prompt and one giant model call. This is better for cost, reliability, governance, and speed.
• How do you balance prompt-debt guidance with context engineering, memories, and shared product context? Drew believes shared context is often necessary, but that teams should treat those instructions as perishable and keep iterating on them unless they’re worth formalizing into systems and evals.
• In compliance, where consistency is critical, what should teams do? Drew’s answer was decomposition, decomposition, decomposition. Break tasks into stages with checkpoints so you can inspect how the model got to its result, rather than trusting one opaque end-to-end answer.
• Does DSPy hide too much and make troubleshooting harder? Drew acknowledged that there is a tradeoff. Any framework gives up some flexibility, but DSPy tries to keep the task-spec layer stable while allowing the implementation underneath to evolve.

Another great audience question, and a good one to end this section on, was “There was prompt engineering, now context engineering, loop engineering, fleet engineering, graph engineering, harness engineering, goal engineering. What’s your take on how to navigate these many engineering disciplines?” I’ll let Drew answer that himself, in the video below.

It’s our job to make it weird

Drew is more optimistic than his worries that LLMs are encouraging a monoculture suggests. If the default output of a model is the average of everything it has seen, “It tells us that there’s still a job for us humans,” he said, “which is that it’s our job to push the model out of distribution. We’re the ones that need to make it weird.”

Weird is a strong word, so don’t take it too seriously. (Though I find it interesting that Harper Reed also used it.) The way I make this point is to say that AI is a medium, like painting or writing or music. Everyone gets the same paints and brushes, the same words, the same notes, but some people draw more out of them than others, or do it better. Our job is to draw something more, something better, out of the ocean of possibilities in the collected knowledge hidden inside an LLM.

But there’s a more prosaic way to push the model out of its normal distribution. Be aware of its training, which is another way of saying “its biases,” and compensate for them. As an example of how to do this, Drew said his team deliberately chose not to use React for a new front end, because the models are trained so heavily on React that using it makes your site look like everyone else’s. He has also started using GLM and Kimi not to save money but because they are more malleable and take direction better inside a custom harness.

That led us into a bit of discussion about open source AI, which is the subject of my next AI Codecon. Drew’s ideas fit right in. He wants the open-weight ecosystem to survive precisely so that models stay infrastructure rather than, as he put it, becoming appliances.

AI is changing the pace of open source development and the security challenges that come with it. Maintainers are reviewing unfamiliar contributions, managing new attack surfaces, and responding to vulnerabilities with limited time and resources.

Session 4 of the GitHub Secure Open Source Fund tested a practical response. The Secure Fund invested more than $500,000 across 50 projects, pairing maintainers with GitHub Security Lab experts, GitHub security tools, AI-assisted workflows, and a peer community.

One lesson emerged consistently: AI can help maintainers investigate, prioritize, and respond faster. Maintainers still provide the context, judgement, and accountability required to decide what ships.

OpenClaw was invited to participate in Session 4 because it is GitHub’s fastest-growing open source project, and its maintainers wanted to strengthen its security posture.

By the end of Session 4, OpenClaw developed an incident response plan, expanded its use of GitHub security tooling, audited its GitHub Actions workflows, and strengthened its processes for identifying and responding to security issues.

The maintainers shared:

OpenClaw’s experience reflects the broader story of Session 4. While the specific risks varied across the cohort, maintainers shared a consistent need: the knowledge, tools, and expert support to secure software as AI changed how they built it.

Across the program, maintainers turned that support into concrete security improvements. Projects strengthened established practices, prepared for emerging AI-related risks, and explored how tools like GitHub Copilot could support vulnerability triage, threat modeling, code review, and remediation.

The benefits extend beyond individual projects. When maintainers strengthen the security of widely used open source software, they help build a more resilient ecosystem for everyone who depends on it.

How the GitHub Secure Open Source Fund works

The GitHub Secure Open Source Fund links funding directly to measurable security outcomes. The program combines hands-on security education, direct engagement with GitHub Security Lab experts, and a trusted community where maintainers can work through security challenges with their peers.

Each session is a three-week sprint and engagement for a total of 12 months. Funding and participation are tied directly to outcome‑driven goals and verified security improvements.

The sprint is designed and curated by the GitHub Security Lab, and delivered by security experts from GitHub and our partners. The training is structured into different focus areas per week.

These include:

• Foundations of open source security
• Threat modeling and secure coding
• AI security and vulnerability management

Throughout this program, each project receives $10,000 USD via GitHub Sponsors (which breaks down to $6,000 USD during the sprint and $2,000 USD at six- and 12-month security check-ins). Projects are invited to a new security-focused community and office hours with the GitHub Security Lab, which they can take advantage of during the full 12 months. They also receive security resources to immediately implement in their project and Azure credits for cloud infrastructure.

• Learn more about the Secure Open Source Fund.
• Apply for Session 5 of the GitHub Secure Open Source Fund before August 24.
• Become a Funding or Ecosystem Partner of the GitHub Secure Open Source Fund.

Where security work happened in Session 4

Session 4 focused on improving security across the systems developers rely on every day. The projects below are grouped by the role they play in the software ecosystem.

AI, machine learning, and intelligent systems 🤖

Caracal • Deep Agents • DocsGPT • LadybugDB • LangChain • n8n-MCP • Nasiko • ONNX • OpenClaw • PageIndex • Scenic • Serena

These projects sit at the intersection of AI, automation, data infrastructure, and machine learning. They increasingly serve as foundational components for modern AI workflows and production deployments. As AI adoption accelerates, security improvements in these projects help establish stronger foundations for emerging AI ecosystems.

---

Build systems, supply chain, and release tooling 🧰

browserslist • CycloneDX Python Library • Cucumber • golangci-lint • JReleaser • postcss • Task

These projects help developers test, validate, package, release, and maintain software across diverse environments. Tools in this group influence everything from software bills of materials and release pipelines to code quality and testing automation.

---

Core programming languages, runtimes, and foundational libraries 📚

Byte Buddy • core-js • FS2 • Gleam • htmx • Pkl • Pyodide • termcolor

These projects help define how software is written, configured, executed, and extended. Improvements at this layer flow downstream to thousands of applications and developer ecosystems.

Security improvements in foundational runtimes and libraries can extend downstream to the many tools and applications that depend on them.

---

Developer tools and productivity platforms ⚒️

cheerio • Ciphey • CodeRunner • Hoppscotch • MapStruct • Python Pillow • Proyecto Respira • Readest • ToolJet • Vuetify • Yjs

These projects shape the everyday experience of building, testing, collaborating on, and using software. Many serve as widely adopted utilities, applications, and platforms that appear throughout developer environments and application stacks.

Together, this group supports API development, low-code platforms, collaborative applications, content processing, and software delivery workflows. When infrastructure projects become more resilient, the benefits extend far beyond a single application and strengthen entire technology ecosystems.

---

Web, networking, APIs, and infrastructure services 📊

actix-web • aiohttp • Apache Solr • Apache ZooKeeper • etcd • FastAPI • Haraka • Hummingbird • mimetype • Sniffnet • Starlette • UAParser.js

These projects form part of the internet’s operational backbone. They handle APIs, networking, search, messaging, service coordination, and distributed systems infrastructure relied on by organizations around the world.

This group includes technologies that sit on the critical path of modern cloud applications and internet services.

---

AI security as a shared frontier

AI-related security questions appeared across projects in Session 4, from machine learning infrastructure and agent frameworks to developer tools and internet infrastructure.

At the same time, established security responsibilities did not go away. Maintainers still needed to manage vulnerabilities, secure dependencies, protect release workflows, and prepare for incidents. AI introduced new risks and increased the speed at which maintainers needed to understand and respond to them.

The lesson from Session 4 is clear: AI security is not evolving in isolation. It is becoming part of the broader practice of building secure software. As that shift continues, maintainers will need practical education, trusted communities, and expert support that can evolve with them.

Thank you to all of our partners

We couldn’t do this without our incredible network of partners. Together, we are helping secure the open source ecosystem for everyone!

Funding Partners: Alfred P. Sloan Foundation, American Express, Chainguard, Datadog, Herodevs, Kraken, Mayfield, Microsoft, Shopify, Stripe, Superbloom, Vercel, Zerodha, 1Password

Ecosystem Partners: Atlantic Council, Ecosyste.ms, CURIOSS, Digital Data Design Institute Lab for Innovation Science, Digital Infrastructure Insights Fund, Microsoft for Startups, Mozilla, OpenForum Europe, Open Source Collective, OpenUK, Open Technology Fund, OpenSSF, Open Source Initiative, OpenJS Foundation, University of California, OWASP, Santa Cruz OSPO, Sovereign Tech Agency, SustainOSS

The post What 50 open source projects taught us about security in the AI era appeared first on The GitHub Blog.

DeepSeek on Thursday open sourced the DeepSeek Harness, a new agent runtime for developers.

The Node.js-based harness is now available on GitHub as a developer preview and under an MIT license.

Clearly, there is some interest in the new harness. Within only a few hours, the repo picked up more than 33,000 GitHub stars, and that number is climbing quickly. Theirs is already a thriving ecosystem of community plugins.

What sets the DeepSeek harness apart is that “everything is a plugin,” as DeepSeek puts it. And the team takes that quite literally. The model adapter, the tool registry, the session log, and the agent loop itself are all plugins — and each one is replaceable.

There’s “no privileged core to patch,” the project’s documentation notes, so extending the harness simply means mounting a plugin beside the others.

The overall architecture is based on Cordis, a “meta-framework for spatiotemporal composability.” To some degree, that sounds more complicated than it is. The core idea here is that software tools like modern AI harnesses require dynamic composition, meaning it needs to be easy to add and remove components without affecting the rest of the system — and those components need to be able to easily interact and understand how they depend on each other.

A recent paper by three researchers from Peking University and DeepSeek explains this in more detail and forms the basis of Cordis and the DeepSeek Harness.

Four modes, one model

The harness itself ships with four presets. Standard mode gives developers the full coding agent, with filesystem tools, shell access, web search capability, subagents, and a plan mode. On the other end, Minimal strips this down to only two tools, bash and str_replace_editor.

Then there is Code mode, which changes how tools reach the model. Rather than exposing those tools as individual function calls, it generates a TypeScript SDK and lets the model write a program against it, so a sequence that would otherwise take five round trips runs as a single call.

The fourth preset, Creator mode, is meant for developers who want to create custom agent presets. It inherits all of the features of the Standard mode and adds runtime inspection, plugin experiments, and preset-authoring guidance.

Everything the model sees gets logged

Another core feature of the harness is that it keeps an append-only session log. Anything that reaches a model request has to be reconstructable from that log.

This also means that the conversation history is not just an implementation detail. Instead, core features like resume, fork, replay, transcripts, telemetry, and the web UI are all based on this single event stream. Adding any new kind of model-visible input therefore means adding a new session event.

The agent sandboxing is similarly strict — as it should be for any agent harness. The local backend wraps subprocesses in Linux Landlock through a Node addon DeepSeek wrote, macOS Seatbelt, or a Windows ACL restricted-token runner.

Someone else’s model

It’s worth stressing that nothing in the harness ties it to DeepSeek’s models. Indeed, the provider catalog covers Anthropic, OpenAI, AWS Bedrock, Microsoft Azure and Google’s Gemini Enterprise Agent Platform (though the documentation still calls it Vertex), alongside DeepSeek’s own endpoint. There is also the option to add custom OpenAI-compatible gateways for other inference providers.

What’s interesting is that the harness also ships with two subagent providers to delegate work directly to Anthropic’s Claude Code and OpenAI’s Codex, resolving each product’s binary from the host PATH so the user supplies the install and the login. Both ship switched off by default.

DeepSeek also ships bridges that run a user’s existing hooks.json from either product against the harness’s own interception points, which the README describes as a compatibility path rather than the better design.

There’s also an MCP client, Agent Client Protocol support, and the harness can read AGENTS.md and CLAUDE.md files.

No pull requests (yet)

As for contributing to the project, DeepSeek currently notes that “We are sorry that we cannot accept external pull requests at the moment.” Instead, it points would-be contributors to GitHub Discussions and to building plugins instead.

The company says it doesn’t consider packages in the official repository inherently more important than community ones, and that readers should treat the repo as “an idea, an official showcase, and a source of inspiration, but not a mandate from us.”

At this point, harnesses are everywhere, of course, and every major lab ships a coding agent now. Among the Chinese labs, Alibaba’s Qwen Code and ByteDance’s Trae Agent both launched in mid-2025, with Moonshot’s Kimi CLI following that October, and Zhipu’s ZCode arriving in July.

DeepSeek clearly believes that its plugin architecture is the major differentiator here, though, and while most of the competition is open source, the early reaction to the new harness shows that even though there’s plenty of competition, DeepSeek may have hit upon something here that developers are looking for.

The post DeepSeek open sources an agent harness where everything is a plugin appeared first on The New Stack.

The trickiest part of shipping an API change isn’t the code. It’s finding everyone who calls your endpoint before you break them. Rename user_id to userId on a GET /users/:id response and your endpoint compiles, your tests pass, and somewhere else in the org a mobile team’s login flow silently returns null on Tuesday morning.

Coding agents don’t help much here. They can read the repo in front of them, but they can’t tell you which mobile app fetches this endpoint at boot, which nightly job depends on the response shape, or which partner integration is contract-tested against the old field name.

That’s what the Postman AI Engineer works on before it writes code. It walks the Postman Context Graph to find every downstream consumer of a change, then opens PRs across every repo that needs to move together.

Why multi-repo API work is hard

In any organization past its first few services, one team’s API is another team’s dependency. A /customers endpoint gets called by a billing worker, a customer support UI, a mobile app, and a partner-facing collection you forgot existed. Rename a field and any of those consumers can break without a compile error.

Code search only sees one repo at a time. A grep for the endpoint path works if every consumer literally hardcodes the URL, but most services abstract HTTP calls behind clients, gateways, or generated SDKs. The dependency exists at runtime, not in the source tree, which is why refactors like this usually turn into a week of Slack messages instead of a code change.

The Context Graph fixes this by keeping a live index of API relationships the way they actually run, not the way they appear in a single codebase. It indexes both your Postman artifacts (APIs, collections, specifications, environments, workspaces, monitors) and the source code in your connected repos, on the backend and the frontend. When the graph tells you who calls /customers, it’s pulling from real code and real requests, not from a wiki page.

What the Context Graph knows about your APIs

The graph is a purpose-built knowledge graph for API dependencies. Every artifact in your workspace and every connected repo shows up as a node:

Node type	What it represents
API	A service, usually with an OpenAPI specification attached
Postman Collection	A set of requests that call an API
Environment	Variables and secrets scoped to a stage
Monitor	A scheduled run that validates an API
Mock server	A stand-in for an API used during development
Workspace	The container that owns a group of APIs
Backend service	A repo that implements one or more endpoints
Frontend app	A repo that calls one or more endpoints
Route handler	A server-side function bound to an endpoint
API call site	A client-side or backend-to-backend call to an endpoint

Edges connect these nodes based on real behavior. A route handler in a backend repo has an edge pointing at the API that owns the endpoint it serves. A fetch('/users/' + id) call in a React component has an edge pointing at the same API. A collection with a request against GET /users/:id gets the same edge. A monitor that runs the collection has an edge pointing at the collection. A workspace has an edge pointing at every API and collection it contains.

The graph is continuously updated as code lands and specs change, so it stays honest about what your systems actually do, not what a wiki page said they did in 2023.

A rename that used to take a week

Here’s the classic scenario. You want to rename user_id to userId in the response of GET /users/:id because you’ve moved to camelCase for new fields, and this endpoint is finally on the list.

Without the graph, this is a week of Slack messages. You post in #eng-general, wait for owners to raise their hands, chase the ones who don’t respond, and hope you caught everyone before the change hits main.

With the Postman AI Engineer, you hand it the task and it queries the graph first. The traversal looks something like this:

1. Find the API node for /users
2. Find every backend route handler that serves GET /users/:id
3. Find every call site in frontend and backend repos that hits GET /users/:id
4. Find every Postman Collection with a request against GET /users/:id
5. For each consumer, walk to the owning repo, workspace, and team
6. Find test scripts, unit tests, and integration tests that assert on user_id
7. Find every mock server that stubs GET /users/:id
8. Find every monitor scheduled against those collections
9. Return the impact set with owning teams and consumer classification

The output is a real list, not a guess. In a typical org, it looks like this:

• **user-service backend repo** (owned by the identity team): route handler GET /users/:id returns a UserResponse DTO with a user_id field
• **mobile-app frontend repo** (owned by the mobile team): three call sites parse response.user_id in the login and profile flows
• **web-dashboard frontend repo** (owned by the frontend team): one call site reads the field into a TypeScript type
• **billing-worker backend repo** (owned by the payments team): a nightly job joins on user_id from the response
• **mobile-app-qa collection** (owned by the mobile team): asserts pm.expect(response.user_id).to.exist in three tests
• **partner-integration-suite collection** (owned by DevRel): contract-tests the field name against a JSON Schema

Now you know exactly which teams to talk to and what each of their consumers does with the field. The dread part of the refactor is done before you’ve touched a line of code.

Making the change across repos in one run

Here’s where the AI Engineer earns its keep. Once it has the impact list, it can propose changes across every affected repo in a single run.

You kick it off from Postman Agent Mode with a prompt like this:

Rename user_id to userId in the GET /users/:id response.
Update this response, the frontend code, the backend code, and anything else that needs to be updated. 

The AI Engineer runs in a sandboxed cloud environment with access to your connected repos through Native Git. It pulls each repo, edits the code and specs, updates any collections stored in the repo, runs the local tests, and opens a PR against the default branch. Backend, frontend, and Postman changes all move together.

What I like most is that every PR description carries the graph context that triggered it. You don’t have to reverse-engineer why the AI Engineer touched a repo. The description says something like:

This repo owns the mobile-app-qa collection, which asserts on user_id in three test scripts. Updating those assertions to match the new field name in the response.

The test scripts get updated in the same PR:

// Before
pm.test("Response contains user_id", () => {
    const body = pm.response.json();
    pm.expect(body.user_id).to.exist;
    pm.expect(body.user_id).to.be.a('string');
});

// After
pm.test("Response contains userId", () => {
    const body = pm.response.json();
    pm.expect(body.userId).to.exist;
    pm.expect(body.userId).to.be.a('string');
});

Because collections are now versioned alongside code in a diff-friendly YAML format, the reviewer sees the assertion change the same way they’d see a source code change. No opaque JSON blobs.

The consumer classification is what makes this safe

Not every downstream consumer carries the same risk. A mock server pretending to be your API doesn’t care about your rename until a real client hits it. A contract test in a partner integration cares immediately. A monitor cares tomorrow morning when it wakes up.

The AI Engineer classifies each consumer before proposing a change. You can see the classification on each PR description, and it looks something like this in the audit trail:

{
  "consumer": "partner-integration-suite",
  "owner_team": "devrel",
  "consumer_type": "contract_test",
  "urgency": "blocking",
  "requires_coordination": true,
  "auto_updatable": false
}

Contract tests and monitors get flagged as blocking. Mock servers and internal QA collections get flagged as auto_updatable, meaning the AI Engineer opens a mechanical PR without waiting for a coordination cycle. That distinction is what lets you actually ship a cross-repo rename in a day instead of a sprint.

Try this against your own workspace

Open Postman Agent Mode in a workspace that has a few APIs and collections in it, and give it a scoped, read-only task first:

Show me every downstream consumer of GET /orders/:id.
Group them by owning team and consumer type.
Don't make any changes yet.

You get back a list of collections, tests, monitors, and mocks that touch that endpoint, with the owning workspace for each. If your APIs and collections live in Postman, you’ll see the same kind of dependency map I walked through above.

From there, escalate to actual changes. Ask it to rename a field, add a required query parameter, or deprecate an endpoint. It’ll walk the graph, propose PRs, and wait for you to approve.

Gotchas I’ve hit

The graph is only as good as what you connect to it. Repos that aren’t connected to Postman aren’t in the graph, and neither are call sites hidden behind a completely dynamic URL builder. Standardize on connecting your service and app repos, and on collections for contract tests, so the impact map matches reality. Every repo you connect makes the next multi-repo change safer.

Auto-updatable is not the same as auto-shipped. Even mechanical rename PRs land in review. The AI Engineer never merges for you. That’s a design choice, not a limitation. You want a human on the final button when a change touches five repos.

Read the classification before you approve. I’ve caught myself skimming a PR title and reaching for merge, only to notice the classification says the change is blocking on a partner team I hadn’t looped in. The consumer classification is in the description for a reason.

Native Git changes the picture. When collections and specifications version alongside your code, the graph updates on every push. Without it, you’re relying on the cloud copy of the collection to stay current, which works but adds a step you have to remember.

Where this goes next

Multi-repo API changes have always been a coordination problem more than a code problem. What the Context Graph does is push the coordination into a place where an agent can actually reason about it. The rename I walked through used to take a week of Slack. It now takes a run.

If you want to see the graph in action, pick a real change you’ve been avoiding because of the blast radius. Ask the AI Engineer to map the consumers first, and see what comes back. It’s usually more (and different) than you remembered.

Resources

• Introducing the AI Engineer on the Postman blog
• Managing downstream dependencies with the AI Engineer
• Why context beats code alone for spec drift
• Postman Agent Mode overview in Postman Docs
• Native Git in Postman Docs
• Postman AI features overview in Postman Docs
• The New Postman is here for context on Git-native workflows

The post Multi-repo API changes with the Postman AI Engineer appeared first on Postman Blog.

Code review is becoming the most important decision-making surface in software engineering, and it’s outgrowing the diff. The assumption was always that code review exists to catch bugs. Its real job is taste, judgment, and applying organizational processes: Is this the right thing for our product?

Now that AI writes thousands of lines of code in minutes, and there’s no way engineers can keep up with the review, engineering teams are stuck between two bad options: skip code review and risk shipping slop, or keep reviewing everything and become the bottleneck. That’s why conversations about reviewing AI code keep going in circles – everyone is arguing about a different part of code review.

The three jobs that remain

Strip away line-by-line inspection, and code review still does three things that teams need.

First, it’s a place to collaborate. This is where teams come together to decide what belongs in the product.

The second role of code review is alignment and knowledge sharing. Every review builds shared context about the changes happening to a code base and the underlying business needs. This context is increasingly important for both people and agents and carries the institutional knowledge and historical considerations for the product and organization.

Its third function is verification: Is the code correct? Will it work? What’s the risk?

“Code is the medium, but the review is where we exercise judgment, and it’s still important.”

These three jobs don’t go away when AI generates the code. If anything, they become more important because the volume of code demanding review is growing faster than the number of people available to review it. Code review was never strictly about reviewing code. Is this even the right name for it? Code is the medium, but the review is where we exercise judgment, and it’s still important.

Planning and review are merging

Code review is where engineering teams conclude that they’re confident this is the right thing for their product. 

As it becomes cheaper to write and rewrite code, a lot of the things teams used to front-load, such as PRDs or architecture documents, are collapsing into the same space as code review. Teams used to do all of that in advance because writing code was expensive, making it important to catch issues earlier in the pipeline. But now they can collaborate on the genuine artifact and shape it until, as a team, they can decide this is what they want. There may be a world where there are 20 times as many PRs that don’t get merged. And maybe that’s how people build software in the future: by trying things, accepting some, rejecting others. Or, teams move the code review process left, before any code is written, by capturing intent and the decision-making process from the LLM session where they happen. The decisions the engineer made while talking to the agent are already there. The discipline is in preserving them before submitting the change rather than letting them disappear.

The reviewer’s job shifts accordingly. Instead of reading a 600-line diff and asking, “Does this look right?”, they read eight lines of intent and ask whether we are solving the right problem with the right constraints. That is a better use of a senior engineer’s time, and it is where the knowledge-sharing function of review survives—reviewers reading acceptance criteria are reading the decisions behind the implementation, not the implementation itself.

“Instead of reading a 600-line diff and asking, ‘Does this look right?’, they read eight lines of intent and ask whether we are solving the right problem with the right constraints.”

That collapse puts more pressure on the code review, but it’s fundamentally the same kind of decision-making. Code review is becoming the place where product decisions get made, not just where code gets checked. Not reading code is a sliding scale

The future of code review is also the center of the debate over whether engineers should still read every line of code, but the framing is wrong.  It’s not “Do I read or not read the code?” It’s  “Does my attention get directed to where it’s actually useful?”

The question is whether reviewers can be pointed at the parts that are meaningful for them to apply their judgment and expertise, while the things that can be removed from their attention set with high confidence fall away over time.

Different organizations will find themselves at different places on that scale. It depends on the sensitivity of what you’re building, the consequences of the change, and what happens if something goes wrong. A team shipping an internal dashboard and a team shipping medical device firmware will make different choices, and both can be right.

AI code needs AI slop register

The standards and correctness checking role of code review is the most likely candidate for AI automation, but not in the most obvious way. Pointing an LLM at a diff with a checklist does not create a quality gate. One of the most underexplored opportunities is closing the loop between review decisions and automated enforcement. Today, a senior engineer catches an API design anti-pattern in review and leaves a comment. Tomorrow, that same anti-pattern shows up in a different PR, and someone has to catch it again. This is where we can and should leverage AI and beat AI slop with AI.

“This is where we can and should leverage AI and beat AI slop with AI.”

When teams actually pull their last 1000 PR review comments and sort them into three buckets (deterministic, execution-testable, genuine judgment), you might end with a split around 45/30/25. Three-quarters of review feedback may be codifiable. The work to codify it is real, but it is finite. Extract those decisions back into rules that become part of the institutional memory of the code base, your AI slop register. An LLM can check, at the time code gets built, with rigor that would have been extraordinarily difficult for humans to maintain—keeping an entire set of rules in your head and applying them to every API was never realistic at scale.

The tighter that loop and the closer it runs to where engineers are already iterating and collaborating, the more effective it gets. 

Stop calling it code review

Code review may not be the right name anymore, but the practice it represents isn’t going anywhere. The code is one of many artifacts we’ll use to assess a change and may become less central as a result. Engineering teams still need a place to collaborate, share knowledge, and apply judgment about what they’re building. That place is evolving, away from the diff, toward intent, toward institutional memory.

The post Code review is a taste problem appeared first on The New Stack.

A few weeks ago, I participated in a 24-hour AI hackathon where we built a product using AI. Necessary tools were provided, a large number of engineers participated eagerly, and a few business folks also joined to bring their ideas into a real-world product using AI.

During brainstorming, people drafted end-to-end process flow diagrams and started working on development, using all the recent available models and platforms to build their product. When the development time window ended, it was time for presentations. As I watched each team present their results, I observed that they couldn’t automate the end-to-end process flow. What they had planned during brainstorming didn’t turn out to be a complete, end-to-end product. Most of the solutions followed the same pattern: they did something in one product and took the output to another product, and the output of that product went somewhere else to finish the loop.