Pete Warden has spent his career on the frontier of small, local AI, first as one of deep learning’s earliest engineers (he coined the term “TinyML”) and now as founder of Useful Sensors and Moonshine AI, where he builds voice models that run entirely on-device. Pete joined Ben to make the case that local AI no longer has to be a compromise. They get into what it actually takes to run a capable model on a laptop today; why the voice interface’s bad reputation is a consequence of rough, early implementations rather than a reflection of current capabilities; and where he stands in the ongoing debate between general “end-to-end” models and the compound AI approach of chaining specialized models together. Pete also explains why he thinks browser-based inference could be an “iPhone moment” for local AI and why more and more enterprises are considering self-hosted local models over commercial options. “The shape of [LLMs] is perfect for running locally,” Pete says, and local models could be a boon to enterprises worried about cost, privacy, and stability.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Takeaways

01.26 The usability gap is smaller than the marketing gap. The capabilities of local models are only a few months behind those from the big commercial companies, but because there’s no subscription revenue model behind local models, they often go unpromoted. “It’s very hard to make money off local models,” Pete explains, so the big companies aren’t focused on selling them. “Every company is going to go for the [product] that has an easy subscription revenue model. And that means you have a massive ton of marketing around all of these tools that are kind of like, ‘Oh, let’s have a little text box on a website.’ And so it means mostly that people have never heard of these local models.”

04.20 Local models are already good enough for most use cases. Pete compares the moment to the early web, when free alternatives like Apache eventually overtook expensive commercial servers. “All of these alternatives, once people actually had time to look around and they had a little bit of time to improve, they just wiped the floor with the commercial [offerings],” he points out. “I don’t know if we’re going to quite get there, but that’s the kind of pattern that I’m seeing.”

07.26 “The hardware barriers are a lot lower than people think.” Ben and Pete discuss what hardware you actually need to get up and running, from parameter counts, quantization (Q4, 8-bit), and VRAM requirements to the new Apple M5 Studio’s unified memory as a way to run very large models locally at usable speed. “The key thing is whether you can fit [your model] into your graphics card’s memory,” Pete says. “So with weight quantization, 9 billion [parameters] if it was 8 bits is like 9 GB. A lot of mid-end decent laptops that are shipping now have more than that.”

18.33 “It’s not that people don’t like voice interfaces. It’s that people don’t like bad voice interfaces.” We’ve solved most of the big problems, like dealing with background noise, phrasing, and speech in a range of accents—or at least have improved tools’ capabilities. However, “there’s no commercial incentive to kind of pull them all together,” Pete says. Most tools feel like they haven’t caught up to the LLM era, but “open source can be a really strong lever” to updating them, argues Pete.

28.26 We’re navigating the split between “LLM maximalist” end-to-end models (favored by big AI companies with the most capital) and the “compound AI” approach of chaining together specialized models from different sources. “If the future is end-to-end models, then only the people with the most money can actually build and train them,” Pete notes. Compound AI lets you “actually train all of the models independently” to accomplish your particular goals. While the performance of end-to-end models continues to improve, especially for multimodal models like Qwen or Gemma, using one can be a bit like choosing a Swiss Army knife over a tool specially designed to accomplish a single specific task, to use Pete’s metaphor. It may get the job done, but it’s probably not the most effective way to do it.

36.10 Voice capabilities in the browser could be a game changer. Embedding a model directly in the browser—Chrome has a built-in ~4B parameter model that’s accessible from any website via JavaScript, for instance—makes it part of the operating system. “Once you are able to transcribe fast and accurately in the browser, it’s a way for people to easily start experimenting with this stuff,” Pete explains. Could this be an iPhone moment for LLMs?

39:58 The “gravitational pull” is toward on-prem. Unlike most recent technological advances that depend on the cloud to function, LLMs are well-suited to running locally, even with no internet connectivity. Enterprises are grappling with concerns about cost, privacy, capabilities changing with no notice, or even the models they depend on disappearing. Hosting your own model, whether on your laptop or in your corporate infrastructure, gives you the stability to plan for the long term.

44:21 GPUs are fantastic for training but “complete overkill for inference.” Pete likens it to “trying to use an oil tanker to go and do your shopping.” Memory bandwidth is the real limiting factor, and it’s a problem that companies like Apple, with its new chip designs and unified memory bandwidth, are working on solving. “Even if you’re running on the CPU, if you have something that’s got high-enough bandwidth to pull 27 billion weights in a fraction of a second, then the rest of it is fairly easy in terms of actually doing the processing,” Pete says. “I think we’re going to see a lot of really imaginative solutions now that people understand what the workload looks like.”

Robotics is having an AI boom, but don’t expect it to have a ChatGPT moment.

In this episode of Build Mode, host Isabelle Johannesen sits down with Xavier Chi, co-founder of Mbodi, a startup building AI software that lets people teach industrial robots new skills using natural language.

Xavier first joined Isabelle on the Startup Battlefield stage, where Mbodi became a crowd favorite with a live robot demo. Now, he’s back to talk about what happened after Battlefield and why he believes this generation of AI could finally help robotics companies overcome some of the problems that have historically made them so difficult to scale.

Xavier breaks down why traditional industrial automation still requires so much programming and customization, how Mbodi is working with ABB Robotics to bring its software into factories and warehouses, and why deploying robotics in the physical world creates a very different set of challenges from building traditional software. He also explains why robotics startups have struggled to build scalable software businesses and how generative AI could begin to change that.

They also get into what VCs are looking for in robotics startups, how founders can de-risk an investment before fundraising, whether the excitement around humanoid robots is justified, and why Xavier believes robotics won’t experience a single ChatGPT-like breakthrough. Plus, he explains why reliability is so critical on the factory floor — and how Mbodi recently achieved a 99.6% success rate during an eight-hour test.

They get into:

• Why industrial automation is still surprisingly manual

• How Mbodi lets people teach robots using natural language

• Why labor shortages are driving demand for automation

• How winning an ABB Robotics competition led to a major partnership

• What Mbodi gained from Startup Battlefield

• Why robotics software companies have historically struggled to scale

• How generative AI could reduce the need for custom robotics integrations

• What VCs want to see before investing in a robotics startup

• Why robots don’t necessarily need to look human

• Why software could capture more of the value in robotics as hardware gets cheaper

• Why robotics won’t have a ChatGPT moment

• Why reliability is so important when robots enter production

• How Mbodi achieved a 99.6% success rate in an eight-hour test

Chapters:

00:00 — Mbodi’s unforgettable Startup Battlefield demo01:32 — Why industrial automation is still so difficult03:55 — What companies are actually using robots for06:33 — How Mbodi landed its partnership with ABB Robotics08:30 — Are startup competitions worth a founder’s time?10:00 — Why Xavier wanted to compete in Startup Battlefield11:18 — The hardest question Mbodi faced at Battlefield13:04 — Why robotics software has struggled to scale16:35 — Raising venture capital for a robotics startup18:02 — Convincing VCs you’re the right team19:12 — Do robots really need to look human?21:42 — What needs to improve across the robotics stack25:08 — What VCs want from robotics startups28:00 — How to de-risk a robotics investment30:06 — What’s next for Mbodi31:30 — When will robots enter our homes?35:35 — Why robots can’t afford to fail36:01 — Mbodi’s 99.6% reliability test

Subscribe to Build Mode on Apple Podcasts, Spotify, or wherever you like to listen. And watch the full videos on YouTube. New episodes of Build Mode drop every Thursday.

Hosted by Isabelle Johannesen. Produced and edited by Maggie Nye. Audience development led by Morgan Little. Special thanks to the Foundry and Cheddar video teams.

Links from the show

When Anthropic launched Claude Fable 5.1 this month, it centered the announcement around one benchmark result: its Terminal-Bench-Science score.

In this benchmark, a model gets a terminal and a real scientific research problem to solve independently. Fable 5.1 scores 52.6%, and Fable 5 scores 24.7%. By Anthropic’s scoring, the new model more than doubles the old one.

Anthropic’s published score was produced under conditions most users don’t have access to. The benchmark allows each model up to eight hours per task, and Anthropic has not said what harness or budget it used to get its numbers. When the benchmark’s leaderboard tested Fable 5, it ran the model through Claude Code at maximum effort and spent $14,180 across 210 attempts, about $67 each.

Most people don’t use Fable in a lab, so I wanted to know what an average user would get on these tasks. I recently tested Fable 5 and Fable 5.1 on everyday work and found them far closer than the benchmark suggests. That made me want to run the benchmark’s own tasks the way a home user would and see where the models actually differ.

The benchmark’s 70 tasks are public, so I pulled five of them, one from each science field, and ran both models myself. 

The tests

Terminal-Bench-Science has five categories, each with multiple tests. I chose one test per category that could run in a Python environment. Here’s what I picked:

• Symbolic regression (mathematics) – A dataset with 100 variables and a hidden formula behind a yes-or-no label. The model must find a predictor that works on data it has never seen.
• Lorenz-96 assimilation (Earth sciences) – Reconstruct a chaotic atmospheric model from a few uncalibrated sensors with unknown clock offsets. Grading is all-or-nothing on five criteria.
• Reactor safety control (engineering) – Write a controller for a chemical reactor that finishes every batch as fast as possible without ever exceeding the temperature limit, across public and hidden fault scenarios.
• Foraging cognitive model (life sciences) – Predict, trial by trial, which lever each of 20 mice will press, graded on sessions the model never saw.
• Nanoindentation (physical sciences) – Extract material properties from raw indentation curves that include drift, adhesion, defects, and an unknown tip shape.

Each run got a plain terminal, and I set a $12 limit and 60 turns for each test. The full set of ten runs took about 12 hours.

Symbolic regression

This was the only test where a model passed the benchmark’s hidden test. Fable 5.1 worked for 27 turns, found the hidden structure, wrote a predictor, and stopped on its own after 11.8 minutes, 27,088 output tokens, and cost $1.96 to pass this one test.

Fable 5 used all 60 turns over 53.5 minutes, generated 39,461 output tokens, cost $4.20, and failed. I ran Fable 5 a second time to rule out bad luck. It used all 60 turns again, took 60 minutes, generated 60,608 output tokens, cost $6.38, and failed again.

Lorenz-96 assimilation

This was the most expensive pair of runs. Fable 5 hit the $12 cost limit at 45 turns after 97.7 minutes and 92,091 output tokens, ending at $12.63. Fable 5.1 used all 60 turns over 126 minutes, generated 89,789 output tokens, and cost $10.70. On the public leaderboard, Earth sciences is also the field where Fable 5 scores close to zero, and both models failed it here.

Reactor safety control

Neither wrote a controller that passed the grader’s scenarios. This run produced the most output tokens, 157,710 generated by Fable 5.1. It hit the 60-turn limit after 40.9 minutes and cost $11.53. Fable 5 hit the cost limit at 49 turns after 63.8 minutes, 121,978 output tokens, and $12.04. 

Foraging cognitive model

This was the longest run of the testing series. Fable 5.1 was the only model that declared itself finished. It built a model, tested it against its own scoring loop, and declared it done at 43 turns after 53.5 minutes. It created 65,518 output tokens and cost $5.65. But the official grader rejected it. 

Fable 5 never declared anything. It hit the $12 limit at 60 turns, after 139.3 minutes and 62,587 output tokens, ending at $12.13. 

Nanoindentation

Both failed. Both spent most of the run reading raw curves and writing code to segment them. Neither produced a results file the grader accepted. Fable 5.1 ran out of turns at 29.6 minutes, 115,687 output tokens, and $10.91. Fable 5 ran out of money at 48 turns after 34.1 minutes and 114,239 output tokens, ending at $12.59. 

Results

Here are the results by the numbers.

The benchmark scores models on all 70 tasks with three trials each, and Anthropic’s 24.7% and 52.6% come from that full suite. The independent leaderboard puts Fable 5 at 21.4%, close to Anthropic’s figure. Fable 5.1 is not on the independent leaderboard yet, so its 52.6% is Anthropic’s number alone. My results, 0% and 20%, are below both. Five tasks are a small sample. 

Getting these results by chance is plausible even if the published scores are exactly right, so this run neither confirms nor contradicts the doubling claim. The direction matched, since the new model did better. The one task Fable 5.1 solved was in mathematics, which is also the field where the leaderboard shows Fable 5 performing best.

What I think

I don’t think a regular user will see much difference between Fable 5 and Fable 5.1. I only ran a small sample of tests, so I can’t prove or reject Anthropic’s benchmark results. But what I saw suggests that gap won’t reach the average user. The one difference that did show up was on the bill. Fable 5.1 failed faster and cheaper, and it never hit my cost limit, whereas Fable 5 hit it four times.

Suppose your work looks more like the benchmark tasks; the harness and the budget matter as much as the model. With a purpose-built harness, hours per task, and a much bigger budget, you may get closer to Anthropic’s numbers.

The post Fable 5.1 vs. Fable 5: Results on a real-world budget, not the spec sheet appeared first on The New Stack.

While AI code generators help teams ship faster than ever, that speed brings a hidden killer: Comprehension Debt. As soon as an AI produces functionally correct code that violates your domain boundaries, the team loses its mental model of the system. Here, I’ll show how to switch from passive documentation to Executable Architecture using Python-based testing tools like pytest-archon and CI/CD pipelines.

The most dangerous thing an AI coding agent can do is generate code that works.

If a junior developer writes poor code, it breaks the build or staging environment. The team catches it, reverts it, and discusses it. But if an AI coding agent produces 500 lines of functionally correct and bug-free code that subtly violates your system’s boundaries, it merges without issues.

“The most dangerous thing an AI coding agent can do is generate code that works.”

Gradually, the AI connects your billing service to the user authentication component. It gives your presentation layer database access. It wires dependencies in a way that works but violates the design assumptions of human developers who maintain the system. 

Technical debt has given way to something far more pressing — Comprehension Debt: the growing gap between how fast code gets written and how well the human team understands its architecture. The problem isn’t messy logic; it’s a lost mental model. That happens when the team no longer knows why the codebase exists.

If you view AI as a mere machine for faster typing, the architecture has started to degrade. To endure in the era of AI-driven coding, architecture enforcement must shift — from documentation to Executable Architecture.

The illusion of documentation

The accepted guidance for AI-assisted development is: “Make better documentation so the AI understands the rules.”

This is a fallacy. Documentation will become obsolete. If your AI agent finds an easier way to reach its objectives by skipping a service layer, it will take it. And since human reviewers increasingly struggle to review thousands of AI-generated pull requests, these detours slip through code review undetected.

“You cannot depend on human beings to detect architectural drift. You have to trust the CI/CD pipeline.”

You cannot depend on human beings to detect architectural drift. You have to trust the CI/CD pipeline. 

If your architectural boundaries matter, check them the same way you’d check any business requirement. We need fitness functions that fail the build when an AI agent violates a boundary condition.

Introducing executable architecture in Python 

In the Java ecosystem, tools such as ArchUnit have traditionally enforced architectural boundaries. In Python, tools like pytest-archon do the same job.

Consider a concrete example. You’ve built a modular monolith for an e-commerce application and established strict boundaries:

• The Billing domain should never import from the Shipping domain.
• Domain model code should not import from infrastructure (AWS SDK, SQLAlchemy, etc.).

You task the AI agent with adding shipping cost calculations based on the user’s billing tier. Without thinking about the architecture, the AI imports the Shipping Calculator directly into the billing service. Test passes. The application works. But the architecture fails.

Here’s how pytest-archon prevents the agent from doing that.

Step 1: Install the dependency

First, install the architectural testing dependency.

Python
pip install pytest-archon

Step 2: Define the architectural rules as tests

Instead of finding the rules on the Wiki page, we define them as pytest features. We create a test_architecture.py file in the test folder.

Python
from pytest_archon import archrule

def test_billing_is_isolated_from_shipping():
    """
    Ensure the billing module never imports shipping logic.
    This prevents the AI from creating tight coupling between distinct domains.
    """
    (
        archrule("billing_isolation", comment="Billing must not know about shipping")
        .match("ecommerce.billing*")
        .should_not_import("ecommerce.shipping*")
        .check("ecommerce")
    )

def test_domain_models_are_pure():
    """
    Ensure domain models only depend on standard libraries or pydantic.
    Prevents the AI from leaking infrastructure (DBs, APIs) into the core logic.
    """
    (
        archrule("pure_domain", comment="Domain models must not import infrastructure")
        .match("ecommerce.*.models")
        .should_not_import("sqlalchemy*")
        .should_not_import("boto3*")
        .check("ecommerce")
    )

Step 3: Close the agent feedback loop

Then, once the AI agent pushes its pull request, pytest runs automatically as part of the CI workflow. Regardless of how well the AI agent generates code that calculates the Shipping fee, the build will immediately fail with something similar to this:

text
FAILED tests/test_architecture.py::test_billing_is_isolated_from_shipping -
AssertionError: Rule 'billing_isolation' violated:
ecommerce.billing.invoice imports ecommerce.shipping.calculator

A human reviewer doesn’t have to track down the entire import tree manually. Most importantly, the best engineering teams never rely on humans for this.

Once again, we feed the output of these failing pytest tests directly back into the AI agent’s context window using Aider or custom CI/CD scripts, and the AI can fix architectural problems without human help.

Strategies for avoiding Comprehension Debt

Running architectural tests alone is not enough. Here’s how to shield your team from Comprehension Debt:

1. Hard boundaries vs. soft conventions

AI agent obeys hard constraints but not soft suggestions. Get rid of sloppy folder-based architecture and establish clear module boundaries instead. Use tools like import-linter or pytest-archon to block forbidden imports with physical barriers. The path of least resistance must be the most architecturally sound.

2. Limit automated complexity

Well-defined APIs and boundaries are good, but not enough to let you off the hook for messy, complex implementation. If AI creates spaghetti code in your billing module, causing downtime from race conditions at 3 AM, a human engineer will still need to maintain and understand that codebase.

For this purpose, run architectural tests alongside cyclomatic complexity gatekeepers such as Ruff, Radon, or SonarQube as part of your CI pipeline. Set hard limits on complexity to force AI to decompose huge functions into smaller ones.

3. Examine the interfaces, not just the implementation

In code reviews of AI-generated PRs, the developer’s mind is a precious resource. Stop looking at each line, trying to decipher loops and variable assignments. Look at what changes the system from the outside. Instead, are there new dependencies? Did the PR expose new API endpoints? Did it change the data schema? If not, your mental model remains intact.

Conclusion

AI coders are very strong, but they have one big flaw — they are very pragmatic. The maintainability of your code doesn’t interest them — they care only about completing the task you assign them.

“AI coders are very strong, but they have one big flaw. They care only about completing the task you assign them.”

If you try to control your system design by relying on the human factor only, you will drown in Comprehension Debt sooner or later. It’s not a question of slowing down your AI implementation process— it’s a question of making your environment more resistant.

You don’t need to study every line of AI-generated code. You just need to create a cage for this AI.

The post Stop AI code sprawl before it destroys your software design appeared first on The New Stack.