OpenAI announced its next big model, GPT-6 Astra, and also, by the way, that "the AGI era" is here now.
Today on The Vergecast, we've got an all-star panel to break down the news of the week.
First, senior AI reporter Hayden Field joins us to talk about our supposed AGI era and Nvidia's acquisition of Hugging Face.
Then, our friend Joanna Stern pops in to talk about new Apple CEO John Ternus, old Apple CEO Tim Cook, and most importantly, to speculate on the format of next week's Apple keynote.
Edge AI moves model execution, model IP, customer data, and system authority into infrastructure the customer owns and operates. That changes who must verify the stack before sensitive assets are released.
Edge AI includes AI systems where inference runs on or near the device, sensor, or other local environments where data is produced and acted on, rather than relying entirely on a centralized cloud service. It is chosen for cost, model selection, sovereignty, latency, and disconnected operation.
Edge AI changes the trust model for AI systems
In Cloud AI, separate companies own and attest the hardware, platform, and model weights. Edge AI deployments often place customers in control of more of the AI stack. This changes the security model because the customer is now responsible for establishing trust across the environment where the AI operates.
In Edge AI, attacks such as prompt injection, model tampering, or malicious firmware updates can occur in the same environment that stores the model, customer data, credentials, and access to physical systems. This shifts trust decisions that were previously handled by cloud providers to the customer. Both the model provider and the customer now share risk: the provider’s models run on customer-owned infrastructure, while the customer must protect the systems, data, and models operating in that environment.
What changes with Edge AI?
Customers operate more of the AI stack.
AI systems can be influenced by prompts, retrieval data, agent instructions, and runtime inputs.
Models, credentials, and data can live in environments outside the provider’s direct control.
Traditional software security controls alone are not enough.
What should organizations do?
Verify runtimes using attestation.
Verify AI artifacts using provenance.
Constrain model actions through mediation.
Bind and release sensitive assets only to trusted environments.
Our post on threat modeling for AI systems covers safety and security issues related to the underlying model. These concerns apply to Edge AI as well. Edge AI adds another question: before releasing weights, keys, or data, what evidence shows the runtime and loaded components can be trusted?
Why Edge AI increases exposure
An Edge AI deployment may include models, prompts, agents, retrieval data, policies, local data stores, and update mechanisms running on infrastructure outside the provider’s cloud environment.
This moves sensitive AI assets and decision logic into potentially hostile environments. Attackers may have physical access to devices, local access to model artifacts, opportunities to tamper with retrieval data or tool configurations, compromise model supply chain and more direct paths from model behavior to real-world consequences.
Disconnected Edge deployments cannot rely on live cloud detection, policy updates, or revocation. They must maintain local verification and enforcement when cloud connectivity is unavailable. Risky AI operations should run only where hardware can protect assets and provide acceptable evidence. Otherwise, the operation should be deferred or revalidated.
Why AI changes the security problem
Unlike conventional software, AI models can be influenced by untrusted content while still using legitimate interfaces and credentials. This article focuses on the security response: architectures that constrain model actions and protect the model, credentials, and data around it.
Traditional software executes code developers ship. AI systems can change behavior based on prompts, retrieval data, agent instructions, and other runtime inputs. Protecting code alone is no longer sufficient; organizations must also establish trust in the data, context, and actions surrounding the model.
Prompt injection can change model behavior. Assume prompt injection will occur, whether direct or indirect. Inputs can affect systems just like executable code because the context window itself acts as an instruction surface. Traditional controls such as signed binaries and code integrity checks were not designed to address this risk.
Trusted data is not always safe data. Traditional vulnerability management does not map cleanly to “data as code,” such as a poisoned retrieval document. Origin signatures can prove where data came from, but they do not prove that the content is safe for an AI system to interpret.
AI behavior is not fully deterministic. The same input may produce different outputs, and small context changes can significantly alter behavior. This limits the effectiveness of techniques such as signature detection and fuzzing. Grounding and tuning improve reliability, but they cannot enforce an acceptable risk boundary. At the authority boundaries it mediates, deterministic policy can still constrain actions when alignment, prompt-injection defenses, or content filters fail to stop an unsafe instruction.
Prompt injection, MCP, multi-agent systems, and computer-use agents on Edge expose different surfaces of these problems. Tool calls are delegated authority, agent output is untrusted input, and screen state is input, not authorization.
These characteristics mean organizations cannot rely on traditional software security controls alone. They also need to verify the environment where AI runs and constrain the actions an AI system is permitted to take.
Constrain model actions through deterministic mediation
Model output should recommend actions, not authorize them. A deterministic mediator outside the model enforces policy by allowlisting actions, scoping arguments, limiting frequency, and releasing credentials only when approved. The mediator is a logical boundary, not another model. It may be provided by the platform or integrated by the customer.
In this pattern, the mediator and its credentials are protected and attested. Mediation bounds what the model can do but does not guarantee that every permitted action is safe; high-consequence or irreversible actions require independent approval, an interlock, or fail-safe behavior.
Establish trust before releasing sensitive assets
Organizations must establish trust in the environment where AI runs and in the artifacts that shape AI behavior. Sensitive assets face two theft vectors: at-rest theft from stored artifacts and keys, and runtime theft while a compromised process holds them decrypted. Before release, the verifier asks two questions:
Do I trust this runtime and the platform on which I am about to execute this workload?
Do I trust these components, such as model weights, tool descriptors, agent definitions, and retrieval indexes, because I trust the system in which they were built and delivered?
Attestation answers the runtime question. Provenance answers the component question. Verifier policy requires both. Either question alone leaves a gap: an approved runtime can load a poisoned artifact, while a trusted artifact can run on a compromised platform. In this trust model, the build system that produced an artifact is treated as another runtime whose evidence is evaluated, and the chain continues until it reaches hardware the verifier accepts. Evidence comes from across the hardware, firmware, runtime, model, integration, and customer layers. It serves different relying parties: customers containing model behavior, publishers protecting model IP, and integrators validating the supply chain. The verifier combines that evidence to gate release.
Verify runtime before releasing sensitive assets
This pattern evaluates a runtime by whether it is measurable, can report its state, and matches an approved baseline before sensitive assets are released.
Confidential compute provides one way to do this. Without end-to-end confidential computing, a privileged host or unprotected accelerator path may be able to read or modify decrypted weights, credentials, and data. GPU/NPU drivers and DMA extend the trusted computing base beyond ordinary application-security visibility. Where confidential computing covers that path within the platform’s documented threat model, protected memory and hardware-rooted attestation can support release to an approved runtime. This is designed to protect assets from host access; it does not constrain a steered model’s actions through authorized interfaces. Those vectors need additional controls.
Because system state can change after deployment, release is treated as a renewable lease that expires when fresh evidence no longer matches the approved state. Physical controls address what measurement cannot see.
In this pattern, evidence gates scheduler placement, storage, identity, and credential release. Bind credentials to the approved runtime and action scope to reduce the risk that credential theft enables bulk exfiltration; otherwise, evidence does not enforce trust.
Verify artifacts that shape model behavior
Runtime trust proves only that the platform is acceptable; it does not prove that the artifacts loaded into it are trustworthy. A clean runtime can still execute a poisoned artifact.
That is why artifact trust has to be evaluated separately. Because these artifacts shape model behavior, accepting one into the system is more than data transfer. Creation, distribution, deployment, and use can each introduce a tampered artifact.
Model IP can extend beyond weights to provider-owned components that handle them; those components may warrant the same evidence-gated release.
In this pattern, approved components and updates enter through a trusted build environment; on-site changes appear as measurement drift rather than silently becoming a new baseline.
Each artifact should carry evidence of its origin, build pipeline, and input integrity. The verifier evaluates that evidence before accepting it. Provenance should chain to hardware and be produced inside a measured, policy-approved runtime. Signatures establish origin and integrity but may not show whether the producing runtime met verifier policy. Provenance helps responders reconstruct what happened: which model acted on which data, in what runtime, and under which policy.
Next steps
Bottom line: Edge AI changes the trust model for AI systems. Customers operate more of the stack, AI behavior is influenced by runtime inputs, and sensitive assets run in environments outside the provider’s direct control. Attestation, provenance, mediation, and evidence-based release help establish trust before models, data, and credentials are exposed.
Edge AI pushes security controls into devices, gateways, vehicles, factories, hospitals, retail spaces, and other customer environments.
For depth on the agent case, see our post on defense in depth for autonomous AI agents. Map sensitive assets, the runtimes and artifacts that can access them, and the party responsible for each release decision. Then define the evidence and policy required at every boundary. At the Edge, security should be architectural: anchored at hardware, enforced at every action.
Providing developers the best model for the task at hand has always been our goal. Earlier this year, we made that easier by launching Auto model selection, which reviews your task and matches it to the best-suited model for that task.
Today, we’re introducing Project HydraFusion, a research preview that delivers frontier intelligence through runtime orchestration. It creates a full execution plan, choosing from models across multiple providers to draft, critique and revise, or cascade to more powerful models to complete your task.
HydraFusion fills a key role in our overall strategy to deliver automated semantic routing between local, cloud, and compound models. For developers, that complexity stays behind the scenes: you select HydraFusion like any other model, and it chooses a workflow that balances performance, cost, and latency for each task.
HydraFusion treats workflow selection as an optimization problem. It uses capability signals for reasoning, code generation, debugging, and tool use to select the most efficient execution pattern to meet the quality bar.
For each request, HydraFusion currently chooses one of three execution patterns:
Single. One selected model solves the task directly.
Cascade. An efficient model drafts a solution and a quality gate decides whether to accept it or escalate to a stronger model.
Critique. One model drafts a result, an independent read-only critic from a different model family reviews it (following the same review pattern as Rubber Duck), and the drafting model revises once.
Figure 1. HydraFusion architecture
Each pattern addresses a different quality-to-cost trade-off. Single preserves speed and efficiency when one model can solve the task directly. Cascade gives an efficient model the first attempt while retaining a path to stronger inference when the candidate does not clear the acceptance gate. Critique adds an independent perspective for tasks where review is more useful than another unaided attempt.
In offline evaluations across three agentic coding benchmarks, HydraFusion consistently demonstrated frontier-level quality with substantial estimated cost savings. On TerminalBench 2.1, it improved verified task quality by 4.9 percentage points at 67% lower estimated cost compared with Claude Opus 5.
Let’s dive into the approach, the results, and the benchmarks.
Adaptive multi-model orchestration
Developers already coordinate models manually: choosing one for a task, asking another to review the work, or escalating a difficult problem to a more capable model. HydraFusion brings that familiar process into the runtime. You choose HydraFusion once and stay focused on your task while it manages the models and workflow behind the scenes.
The key is selectivity. Some coding tasks can be solved directly, while others benefit from review, revision, or escalation. HydraFusion evaluates each request and chooses the least complex workflow expected to meet its needs, using additional model calls only when they are likely to improve the result. This adaptive approach balances quality, cost, and latency across models.
As the model frontier advances, so does HydraFusion. When new models become available in GitHub Copilot, we can evaluate and incorporate them into its model pool, bringing their strengths to the tasks best suited to them.
Building HydraFusion
Turning adaptive multi-model orchestration into one dependable coding experience requires careful control of execution, review, cost, and repository state. HydraFusion is built around five operating principles:
Complete accounting. Aggregate cost and usage across every workflow leg, including drafting, critique, revision, escalation, retry, and fallback.
Bounded execution. Give each leg explicit timeout and cancellation behavior to keep execution and cost within defined limits.
Isolated review. Run review steps in isolated, tool-less contexts, while solver steps use the shared workspace and normal permission-aware agent loop. This allows models to assess the work independently without modifying the repository.
Fail-safe application. Apply no patch when the workflow is cancelled or fails validation, preventing incomplete changes from reaching the repository.
Validated routing. Verify workflow definitions, model bindings, fallback behavior, and model availability before execution begins.
Together, these principles make multi-model orchestration practical for repository-level work. Internally, the runtime records the role, outcome, cost, latency, and diagnostics of each leg so the workflow can be understood after execution. Externally, the developer receives one coherent response and one permission-aware change set.
Benchmarking results
Fixed HydraFusion policies were evaluated across three agentic coding benchmarks — TerminalBench 2.1, DeepSWE, and CheckpointBench, our internal benchmark based on real GitHub Copilot sessions — using Claude Opus 5 and GPT-5.6 Sol as comparison baselines. Each policy used the same task inputs, tools, execution limits, pricing assumptions, grading conditions, and treatment of missing results. The evaluation measured verified task quality, which is the share of tasks confirmed as correctly answered, and the complete estimated workflow cost. Cost accounting included every invoked leg, such as drafting, critique, revision, escalation, retry, and fallback. The results below show the best tuned HydraFusion configuration.
Benchmarks
Cost vs. Opus 5
Quality vs. Opus 5
TerminalBench 2.1
67% lower
+4.9 points
DeepSWE
36% lower
-1.5 points
CheckpointBench
65% lower
-0.1 points
Table 1. HydraFusion quality and cost across three agentic benchmarks, relative to Opus 5.
These controlled offline results are specific to the evaluated benchmark revisions, workflow configurations, model pool, and pricing assumptions, with all models evaluated at the same medium reasoning level. Through this research preview, we’ll validate how these results translate to real developer workloads and use the findings to further optimize HydraFusion for production quality, latency, reliability, caching efficiency, cost, and safety.
TerminalBench 2.1
TerminalBench 2.1 evaluates coding agents on complex, multi-step tasks in terminal environments.
Figure 2 compares HydraFusion and Opus 5 across verified task quality and estimated workflow cost.
DeepSWE
DeepSWE evaluates challenging repository-level software engineering tasks that require navigating large codebases, understanding cross-file dependencies, and producing end-to-end fixes. On this benchmark, HydraFusion comes within 1.5 percentage points of Opus 5 while reducing cost by 36%, demonstrating a compelling quality-cost tradeoff for complex real-world engineering tasks.
CheckpointBench
&
CheckpointBench is an internal multi-turn benchmark curated from real GitHub Copilot agentic coding sessions. Each conversation is anchored to a specific public repository and immutable commit, ensuring every session is replayable. The benchmark is balanced across language, task type, difficulty, scrubbed for quality, resulting in a realistic evaluation set that closely mirrors production agentic sessions. On this benchmark, HydraFusion comes within 0.1 percentage points of Opus 5 at 65% lower cost.
Early internal testing has echoed that result.
So far, the reasoning and task solving capability [of HydraFusion] is at or better than Opus.
Principal Software Engineer at Microsoft
Hill-climbing HydraFusion
HydraFusion’s routing policies were shaped by how developers use GitHub Copilot on real coding tasks. To make those workflows reproducible, we curated CheckpointBench from real Copilot coding-session trajectories. We refined HydraFusion repeatedly across CheckpointBench, DeepSWE, and TerminalBench 2.1, optimizing across the evaluation sets rather than for any single benchmark.
HydraFusion’s per-capability scores provided a consistent basis for comparing candidate routing policies. Instead of manually tuning thresholds, we used beam search to build the optimal decision policy. Each candidate was measured against a frozen baseline on quality, cost, and failure modes, so improvements were evaluated on stable ground.
TerminalBench 2.1 provides the most complete sequence of runs, making it the clearest view of this iterative improvement. The progression was not linear. Between August 11 and August 25, two operational failures in the evaluation harness produced invalid runs. Those failures were excluded from the performance trend, corrected, and followed by continued gains in the HydraFusion configurations. By August 25, HydraFusion had reached its strongest operating points in the recorded series.
This development record shows how the policies improved from repeated experiments. TerminalBench 2.1 was one of several benchmarks used during development. Its relative saturation makes broader validation important, so the three-benchmark evaluation also includes DeepSWE’s more demanding repository-level tasks. The research preview extends that learning loop to real developer workloads.
Try the research preview
For this preview, first-turn, single-prompt coding tasks are the best place to start. We’ll be focusing on strong multi-turn performance with longer, iterative sessions next.
This preview is designed to learn which tasks benefit from compound workflows and how orchestration affects latency and cost in practice. For the best experience today, start with substantial, well-scoped coding tasks that you can hand to Copilot in autopilot mode in a single prompt. Share what you find, including where it excels, where it falls short, and what you’d want to see next, through /feedback in Copilot CLI or in the GitHub Community discussion.
HydraFusion remains an active research effort. Results, models, workflows, availability, names, and product behavior may change as we learn from the preview. We believe the next real gain in coding agents will come from combining frontier intelligence with runtime orchestration. HydraFusion is our first bet on that idea: moving from choosing the best model to dynamically constructing the best way to solve each task.
Acknowledgments
A huge thank-you to the researchers, engineers, product managers, and designers across GitHub and Microsoft who curated the training data and built the training pipeline, evaluation suites, client experience, and serving stack. We are especially grateful to the GitHub Copilot CLI, Copilot API and VS Code team for overcoming numerous challenges to bring this research preview to our customers.
As a software engineer with more than seven years of experience before the coding-agent era, I never liked the idea of vibe coding. But I knew there was a clear line between it and using coding agents to generate clean, maintainable code. That line, where good software principles meet coding agents, is defined by software factories.
That’s why, three months ago, I built my own software factory, Squid, to ship all of Decoding AI’s small and medium projects with minimal human intervention. The first version was so overbuilt I stopped using it.
Meanwhile, I kept seeing people obsess over the next “_____ engineering” label, instead of focusing on actionable outcomes. Prompt engineering, then context engineering, then harness engineering. So far, so good. But in the last few weeks (July 2026 as I write this), things got off track with loop engineering and graph engineering, which already read more like marketing talk than like anything that solves real problems. Graph engineering overtheorizes how teams have built AI applications since the LangGraph era kicked off in ~2024. Don’t get me wrong. The terms aren’t wrong (Boris Cherny, who leads Claude Code at Anthropic, says, “My job is to write loops”), but we’re overexplaining intuitive things we started doing years ago.
While you’re defining what counts as a loop, you’re not thinking about the processes that actually deliver software.
I bet you already have an intuitive sense of what a software factory is. In this article, I want to further formalize it and map it onto the software development lifecycle (SDLC). We’ll explore how big your software factory should be, and when to stop automating before it adds more friction than value. Most importantly, I want to highlight where the human belongs in this process, and where I believe they’ll still belong even in a world where all the code is generated by AI.
So…what’s worth automating? Where does the human bring the most value? What’s worth building, and what’s worth buying?
The design of a software factory
Like a physical factory, a software factory automates software creation with minimal human input. Raw work (bug reports, feature ideas, incidents) goes in. Shipped software comes out. It needs a few highly qualified people making high-leverage decisions, and defined gates that work can’t pass through without them.
The software factory line. Eight stages over one shared context layer, with production signals looping back as new tasks.
The factory is made up of eight stages that can be divided into three buckets:
What to build.Triage/intake classifies, deduplicates, and routes incoming work. Brainstorming finds high-impact features through market analysis, user data, and technical research. Planning, the most important stage, turns that research into a polished plan, refines it by letting the agent grill you, and tracks decisions in an ADR (Architecture Decision Record) log plus a glossary. The outputs of this stage are tickets backed by documentation that a team of agents can implement, which can be tracked in plain files or a project management tool, such as GitHub Issues, Linear, or Notion.
At this stage, the agent plans in read-only mode, going through the code, the AGENTS.md file, and most importantly the context layer.
Actual building and checking. Implementing is a software engineer and QA agent loop that goes through the tasks and supporting documentation. Review checks the PR diff against product, architecture, and code standards. Review-CI runs the test suite, and failures trigger a fixing agent. Release handles CD to staging/production with human deployment checks.
Self-improving. Monitor/incident response feeds production signals (alerts, errors, incidents) back into triage as fresh input on what to build next, closing the loop.
Orthogonal to the eight stages, we have the context layer. The layer is especially important at the front of the line. Brainstorming is limited to the data it sees: user analytics, competitor analysis, research, transcripts, and documentation. At this stage, a poor context layer directly limits the space of possibilities you can explore. It has a similar impact on planning, where transforming the raw idea into technical specs and tasks depends heavily on how good the examples in the context layer are. If you want to implement a new product-recommendation feature, and you have zero examples, the LLM will just predict the most common thing to do, which often isn’t the best solution for your product.
The context layer can take many forms. One strategy that is becoming increasingly popular is the LLM Wiki, a term coined by Andrej Karpathy. It’s basically a strategy to transform your data into a structured knowledge base, just by using files, instead of a database. Factory, via its AutoWiki feature, transforms popular codebases into a structured knowledge base that agents can query instead of parsing the codebase itself. LangChain recently released OpenWiki, a CLI tool to manage wikis for agent memory. If you’re curious, in this article I detail how I turn my data from Obsidian, Readwise, and Google Drive into agent memory via LLM wikis.
Where the human belongs
To see where the human belongs, let’s walk through the factory with an end-to-end example. We’ll build a feature for a shopping-assistant agent on an ecommerce platform similar to Amazon’s. The scenario is that usage data says users aren’t engaging with its recommendations, and we have to ship an improvement.
Brainstorm is where taste lives. Agents do the grunt work: They analyze user activity, scan competitors’ assistants, and pull research into the knowledge base. Then a member of the technical staff starts looking at the data, understands why people are not engaging with the recommendations, explores how the competition implements their solution, and proposes a fix as a feature spec. At this stage, the spec solves a business problem. It doesn’t need to prescribe a technical solution yet.
Plan is where a human, with the help of the software factory, transforms the feature spec into an implementation plan. Let’s assume that we want to make a change to the recommender engine algorithm. The human chats with the knowledge base, figures out whether it’s feasible, and thinks through architecture, interfaces, data flow, cost, and latency. They then let the agent scan the codebase and grill them until the plan is properly refined into something that fits the codebase. The output is a bunch of tickets plus an ADR explaining the algorithm change and an update to the glossary.
The agent can help within these two stages by quickly scanning through a ton of data and improving the plan, but the human is still central.
Use the strongest model (Fable) for brainstorming and planning. These stages burn fewer tokens than implementation itself, but everything downstream depends on them. A well-written plan lets cheaper models (Opus, Sonnet) execute without reasoning their way out of dead ends. A weak plan makes them retry until the extra tokens erase the price gap.
With a weak plan, I’ve watched Sonnet on high reasoning out-cost Opus on the same task: The smaller model needs more attempts to reach the same goal. Total cost is tokens × price, not model tier. So more failures equals more reasoning, more tokens, and more cost.
From here on, we move into “loop” and “graph” engineering territory.
A strong planning session makes cheap executors actually cheap, while a weak plan makes them expensive.
Implement runs a software engineer agent that picks up every ticket that’s ready to go. As the loop is scoped to a feature, it takes only the associated tickets. After each ticket is implemented, a QA agent tries to find bugs by stress-testing the application. As agents tend to have a positive bias towards their own work, the split between the software engineer and QA agents matters. As Addy Osmani puts it, the model that wrote the code is “way too nice grading its own homework.” This loop, at solo scale, can be as simple as a bunch of terminals pulling tickets. At larger scale, it runs on remote agents working 24/7.
The loop only works if agents can interact with the app. The QA agent needs one command that starts the whole stack reproducibly. From there, it drives the app in the browser, calls the data or fine-tuning pipelines, or hits your server’s API. Whatever your app’s interface is, the agent needs access to it, the same way a human user would.
The key idea is to integrate feedback loops as natively as possible into your software factory. Ideally, you want multiple levels, depending on how expensive it is to run them: linting, unit tests, integration tests, and end-to-end tests. When the loop keeps failing, the root cause is almost always missing plumbing, not the agents.
Review has three steps. Step one checks the product and architecture requirements against the ticket and the ADR. Any discrepancy becomes a new ticket passed back to the implement loop. Step two ensures code quality (modularity, naming) and guards against AI slop such as verbose comments or cryptic function names. Step three looks at the CI/CD pipeline. At every step, any failure auto-creates a task picked up by the software agent.
Not every project needs all three steps. The “factory” ends with a PR that you as a human need to review and merge. But in reality, if you spend enough time creating a strong plan, the PR that reaches you is usually ready to ship as-is.
The dynamics between what humans and agents own in the software factory line.
So where does the human belong? You’re indispensable during brainstorming and planning, and you return for the final check. Agents own everything in between. OpenAI took this to the extreme: ~1M lines and ~1,500 merged PRs over five months with zero hand-written lines. Their framing is “Humans steer. Agents execute.”
Don’t overbuild the factory
With my first Squid version (my own software factory), I got greedy and chased full autonomy: big remote workflows, parallel agents, and one grand pipeline running end to end. It worked, until something went offscript. Which it usually does. I couldn’t debug it, couldn’t halt it mid-run, and couldn’t redirect it without throwing the run away. It was a big monolith that took me too far out of the loop, and I couldn’t control it.
I realized you need two options. The first is granular commands that let you grill your plan, implement a specific task, or review one particular step. The second, for when you’re comfortable giving the agent 24/7 autonomy, is an end-to-end command that chains all the smaller ones into a fully autonomous graph, such as one big /plan and /implement-review-all command. Basically, each step is a “loop,” while the whole pipeline is the “graph” of your software factory. Still, note how planning and the rest are split into two different commands, as planning is, and always will be, human-driven (at least if you want the result to stay aligned with what you actually want).
Bottom line. You need to be able to step in, halt it, redirect it, and interrupt it, while still having the option to go fully autonomous.
The bottleneck is me, and that’s by design. To be honest, I’ve worked mostly solo since the AI coding agent boom, and I don’t understand who the people shipping 100 features in parallel are. Most of my features (per project) build on one another, which makes them impossible to parallelize. As the project grows, you can find more and more independent features that can be implemented in parallel, but I still believe that number is limited.
That’s why, when I parallelize, I only use local agents, each running in an isolated codebase via worktrees. So far, I’ve never felt the need for 24/7 remote agents, or wanted the overhead of managing them.
A big team can justify more automation, but it has to earn it. So as with any other software product, start small, start by automating the most time-consuming bottlenecks, and add complexity gradually as people get comfortable with the system. Don’t be like me, with my Squid experiment.
Build vs. buy
In all scenarios, you’ll start with a prebuilt coding harness. The most popular vendor-locked ones are Claude Code and Codex. Or go open source with OpenCode or Pi, which took off thanks to its minimalist, extensible architecture that lets you easily build on top of it.
But picking a harness isn’t the same as knowing how to configure it and wire it into your software factory. That’s why everyone needs to know, at least intuitively, how a coding agent works under the hood: the agent loop running in your terminal, what changes when it runs remotely, how you evaluate it, and which context engineering strategies keep it cheap without making it dumber. If you want to learn more about building a coding agent from scratch, consider exploring my open source course on GitHub. Even if you never plan to build your own harness, that intuition is what lets you become a power user.
For a small team, you’ll get extremely far just by defining a set of skills and agents that encode your process on top of the coding harness (a.k.a. your software factory). To keep it simple, this is what I did with Squid, which I use to implement all my projects.
There are other off-the-shelf “software factories” powered just by skills and agents defined in .md files, such as Matt Pocock’s skills repository or the BMad method.
But remember that the factory is mostly about processes, not tools: A factory that doesn’t fit how your team already works adds friction, never gets adopted, and ends up useless.
You cross the buy line the moment engineers you don’t personally supervise run agents. Observability, tracing, cost tracking, and pay-per-token billing stop being optional and become someone’s full-time job. Agent swarms wired into Linear, Slack, and CI across distributed infrastructure are a logistical hell that isn’t your product. That’s when it makes sense to look into off-the-shelf solutions such as Factory.ai (that comes with the Droid agent), or Warp’s Oz. In Warp CEO Zach Lloyd’s words, “Most of the factory is not necessarily a new interface. It is an integration into people’s existing workflows.”
At the other end of the spectrum, you cross back to building when the platform’s constraints cost more than the team it would take to replace it, as OpenAI’s report on its Codex-built product shows.
The smallest builds, the middle buys, and the largest builds again.
What’s next
Someone is already coining next quarter’s “_____ engineering” term as we speak. But the software engineering processes you use to output real code won’t change that often. That’s why you should be open-minded but at the same time focus on actionable outcomes, not on overthinking how to label things.
The harsh reality is that software factories are just at the beginning. They’re far from perfect, and especially far from being fully “autonomous.” Usually, when someone claims they’ve cracked the software factory problem, they either haven’t tested the idea enough or are trying to sell it to you. I’m confident we’ll reach the point where almost the entire software development lifecycle is automated (with the exception of brainstorming and planning), but at the moment we’re still figuring things out.
But here’s what I’m wondering:
Which stage of your factory still needs you the most? I keep automating mine, and the bottleneck stubbornly stays at planning.
Ryan chats with Greg Jennings, VP of Engineering for AI Products at Anaconda, about what it takes to build a secure-by-default AI coding agent, why prompts shouldn't be treated as strict security guardrails, and how Anaconda is using strategic acquisitions to secure the AI software supply chain.
Log in
