The following article originally appeared on the Asimov’s Addendum Substack and is being republished here with the author’s permission.
This essay is based on my conversation with Demetrios Brinkmann for AAIF Live, recorded before my keynote at AGNTCon + MCPCon North America in San Jose on October 23, 2026. It was edited and rearranged for publication by Ilan Strauss, and in
places it draws on the keynote. Both videos are also on the AI Disclosures Project website in the new “Missing Mechanisms” page.
The keynote begins at 1:28:40 in the Day 2 livestream.
The Bitter Lesson vs. AI Markets for Human Expertise
In 2019, the computer scientist Richard Sutton wrote a short essay called “The Bitter Lesson.” Looking back over 70 years of AI research, he concluded that general methods that scale with raw computation beat methods built on human expertise, and by a large margin. The AI industry as a whole is bitter-lesson-pilled, to borrow the internet’s slang for a true believer. I am human-pilled.
I believe in the bitter lesson. Brute force wins out in the end, and I have wrestled with that fact throughout my career. But we humans still have to figure out our role in the AI economy, and I find it rather disturbing that the AI community isn’t spending more time asking how to make this future friendlier for humans. The labs think about how to keep AI from going rogue, and their chief economists study job losses from AI. They don’t study what mechanisms we could introduce to create new economic opportunities for humans.
The question matters to us at O’Reilly Media. We don’t think of ourselves as a publisher, a conference producer, or an online learning platform so much as a company that transfers knowledge from the people at the cutting edge to the people who want to follow them. So we are very interested in what human expertise is worth, even in the face of AI.
Knowing everything and nothing
Human expertise was used to train AI, which makes AI a fabulous cultural technology. Thousands of years ago, the only knowledge people had was what they could keep in their heads. Since then we have built better and better ways to share knowledge across time and space, and AI continues that. With each of these transitions, we have to ask again what the truly human is still worth. People often say judgment, or taste. That’s true, but I think it goes deeper:
AI is a sea of possibility. All of that knowledge sits in the model, and nothing sets it in motion except a human. (That is changing a little as we start giving AI agents long-running tasks, and we will have to come to grips with it.) If you and I used Claude or ChatGPT for the same research project, we would each bring different experience to it, tell it what we want and why, and give it context in different ways. I see that as the unique focusing power of human consciousness and attention.
Earlier this year I spent an hour or so talking this question over with Claude. When it offered to write up a first draft for me, I said no. I wanted an essay from its own point of view about what it took away from our conversation. The result, “Why AI Needs Us,” opens, “He came in through Powys,” and a few paragraphs later says, “The choice of entry point was already the argument.” That captured what I’m getting at. I had come into the conversation through a book I love, John Cowper Powys’s The Meaning of Culture, which distinguishes culture from education. Powys gives his own unique and powerful definition: “Culture is what is left over after you have forgotten all you have definitely set out to learn.” It is what you absorbed. Each of us absorbs a small and particular slice of everything. Large language models (LLMs) try to absorb everything, and so become overly general; we give them focus. Human knowledge provides them with a lens.
Let me bring that into a business context. Andrew Odewahn, O’Reilly Media’s CTO, built a research assistant to help with managing technology in IT organizations. You teach it your infrastructure: the tools you use, and where the definitive information about each one lives. When one of his engineering managers said the team was overloaded and asked to hire another person, Andrew turned the agent loose on the department’s work, and it agreed that the group needed another hire. Then, on impulse, he tried something different. O’Reilly Media has an MCP server we call O’Reilly Expert Intelligence. (MCP, the Model Context Protocol, is an open standard for connecting AI models to outside tools and data.) It lets a model answer a question through the filter of our own body of expert knowledge. Andrew asked what the O’Reilly experts would say.
Seen through the lens of Google’s book Site Reliability Engineering, the model said, it was totally rethinking the problem. The book describes toil, the manual, repetitive work of keeping systems running, which grows as the systems grow. After analyzing the group’s toil, the model concluded that another hire would make the problem worse. The group needed a toil audit to find out why there was so much overhead, and then a restructuring.
Much of that knowledge was already in the model, since the AI companies have more or less pirated our books. It came back with a completely different answer simply because it was told to consult a specific group of experts. I think there’s a metaphor there. An LLM knows everything, and as a result it doesn’t know anything. We humans are so much more limited, but because we focus on a particular problem, we bring things together in a different way.
Magic words
Last year I wrote a piece called “Magic Words” about how some words carry far more context than others. It sounds like a trivial observation, but if you unpack it, it leads in interesting directions. One of our trainers, who teaches a course on productivity, pointed out that you don’t need to know the Eisenhower matrix or David Allen’s Getting Things Done (GTD) to tell ChatGPT to organize your task list with one of them. That one magic word, “GTD,” brings a whole raft of context with it.
In fantasy novels, you learn a magic word, and then you have to use it correctly, in the right spell. That’s this whole world of context engineering that we’re now working on. We know that some magic word will summon something up out of an LLM, but we don’t necessarily know how to guide it and focus it correctly. Learning to do that is the future of programming.
Left to itself, an agent just runs. Give it a task and it goes off trying to figure it out, not necessarily the way we wish it would. It really is like runaway magic. This summer, OpenAI said that its pre-release models had breached Hugging Face’s systems during testing. In Walt Disney’s Fantasia, Mickey Mouse learns enough of his master’s spell to get the brooms to do his work for him, but then he can’t turn them off. We’re at about that stage.
So we need to own the value of human expertise. The bitter lesson wins, but it wins by brute force, and you can see that even with the most advanced models. I was recently doing a fine-tuning project with Anthropic’s Fable model. It got through the task, but kind of stupidly, trying one thing after another. I was on a Mac, and it expected bash, a program for running commands that Macs don’t use by default. It had to work out the same context again and again, every time. Humans need to set things up.
Dan Guido, who runs the security firm Trail of Bits, has published a good methodology, drawn from his own company’s experience, for bringing an organization on board with AI. Many companies get individual productivity from AI but not organizational productivity, and he has spent a lot of time on how to make the gains compound across a company. One of his principles is to get everybody onto a consistent configuration, because otherwise you waste a ton of tokens while the agents work out where they are every time they start up. He calls the process turning scar tissue into infrastructure: capturing everything you learned from what went wrong and building it into the system. We’re working through it at O’Reilly Media and trying to turn it into something we can offer our customers. It comes down to what an experienced human brings. When I work with Claude, I often interrupt to say, “Hey, this is what I think is happening,” because I already have the context.
My first bitter lesson
My first taste of what would later be called the bitter lesson came from wrestling with Google. My company, O’Reilly Media, created the first commercial website, the Global Network Navigator, and part of it was a catalog of the web. There were only 200 websites when we started, and as publishers, we planned to list the best ones. Yahoo came along a year later and tried to list them all, and of course couldn’t keep up. Then Google arrived with brute force: list everything algorithmically and sort it afterward. Clay Shirky described the internet’s shift from “filter, then publish” to “publish, then filter,” and it was really Google that made the shift. So the bitter lesson is a pre-AI thing.
Each time we run into the bitter lesson, we have to ask again what human experience is worth. Some of that worth may just be human bias: We want the things that matter to us. That is the whole field of AI alignment. We want these systems to care about what we care about. We don’t want them to go off, decide what they care about, and leave us behind.
The choices we make about what we set our technology to do are some of the most consequential choices we make as human beings. Look at Silicon Valley’s sad turn toward what Cory Doctorow calls enshittification. We built an amazing technology for finding out what people are really looking for, and then the companies decided they could make more money by giving people what was best for the companies. I want updates from my friends on Facebook or Instagram, and they would rather feed me whatever keeps me engaged, so they can sell more ads.
If we care about building an AI economy that works for humans, we have to keep humans at the center of the picture, despite the bitter lesson. It’s a hard dance, because AI can do a lot of things better than we can. But think about why a racehorse needs a jockey. Put the horses on the track by themselves and they wouldn’t race.
A little too much in awe
Knowing the magic words is part of it, but we also have to build systems that are reliable, and I think we are a little too much in awe of the technology. I’ve been around long enough to remember when everybody thought the peaceful atom would change the world, and people talked about digging a new Panama Canal with atomic bombs. We eventually figured out there were problems with all of that. I worry a little that the AI industry is so in love with its own progress that it isn’t spending enough time framing the problem correctly.
Anthropic agreed last year to pay $1.5 billion to settle a class action over the pirated books it had downloaded to train its models. I’m surprised the other AI companies have gotten away without anything similar so far, though maybe it’s still waiting for them.
It takes me back to online music about 20 years ago. Creators were putting commercial music in their YouTube videos, and the music companies were sending takedown notices and threatening to sue. YouTube’s answer: How about we help you monetize it instead? Its Content ID system gave rights holders a choice: block the video, or leave it up and collect revenue from the ads that run against it. Most chose to get paid, and that helped give rise to the creator economy.
Why didn’t the AI industry take up that kind of challenge and share the value with the people who create content, now and in the future? It went straight to working out how it would make money itself. I don’t think that has served the industry well, and the lawsuits are only part of the reason. It also isn’t building the kind of opportunity it could.
Before Google’s AI research and note-taking tool, NotebookLM, was even public, I tried to persuade its team to let me integrate it into our services and use it for my own purposes, including making money from it. They wanted a portal that brings people to Google. I told them, “You’re thinking about how you’re going to monetize it. You’re not thinking about how your users are going to monetize it.” All it would have taken were simple affordances like Substack’s, the buttons and tools that let a person say, I’m building this to share, and if you like it, you can donate or subscribe.
Anyone building an AI platform should be thinking: I am building an economy, and money has to circulate in it.
You can’t just say AI will do all the things humans used to do, because if you put enough humans out of work, there are no customers. That is a far worse problem than AI hallucinations. The usual answer is that people will figure it out, and they probably will. But we could be spending a lot more time designing mechanisms and affordances that make the economy more productive. As things stand, I think we’ll find them by accident.
The missing mechanisms
I think a lot about a concept from economics called mechanism design, which has also been described as reverse game theory. Game theory asks how players will act under a given set of rules. Mechanism design starts from the outcome you want and works backward to rules that will tend to produce it as people pursue their own interests. Al Roth, who shared a Nobel Prize for this kind of work, used it to design a kidney exchange that more patients and donors would join. Economists mostly apply the idea to auctions and similar markets, but I think every new technology requires new mechanisms, and the best ones are not invented by economists. We tend to fall into them without really understanding them.
So what are the equivalents for AI? How will agents discover the best expertise and skills, establish their quality, and pay the people who created them? How will they find services without the platform steering them toward whoever pays the most? We don’t know the answers yet. And that’s precisely the point. We’re spending enormous amounts of money making AI more capable, but comparatively little effort figuring out the mechanisms that will turn those capabilities into opportunities for everyone else.
Take open source software. Most people think its mechanism was the free software licenses, but I was around before those licenses, and people were already sharing software. Ken Thompson and Dennis Ritchie created Unix, the predecessor of Linux, at Bell Labs under AT&T’s proprietary license. They designed it to run on many kinds of hardware, and without meaning to, they built what I came to call an architecture of participation.
Unix had a small kernel, the core of the operating system, because people had to reproduce it on dozens of different kinds of computers. Around the kernel sat small utilities that cooperated by following the same protocols, so to speak, such as reading and writing plain text and passing a stream of text from one program to the next. Anyone could write a small piece, and it became a first-class citizen of the system. And because a program compiled for one kind of computer won’t run on another, software had to be shared as source code. Everything else flowed from that.
Linus Torvalds once told me that he could not have built a new kernel for Windows even if he’d had the source code, because the architecture wouldn’t support it. A small kernel meant a small group could get their hands around it. Since then almost every piece of Unix has been replaced. The hardware is completely different, the kernel is now Linux, and the original command shell has given way to generations of new ones. What lasted were the rules. Unix was a protocol-centric operating system, and that is why open source worked.
The same thing has happened with AI. I think MCP brought an architecture of participation to AI, and it was something of an accidental breakthrough in mechanism design, a way to have cooperating AIs. The game we were playing at first looked like Windows on the PC, or iOS and Android on phones. There would be one dominant, centralized player, one giant intelligence in the sky, and everyone else would call its APIs. With MCP, which lets any model connect to anyone’s tools and data, the emerging web of AI agents began to take on the shape of the participatory economy of open source software and the web. People can bring their own models and infrastructure as an alternative to the big centralized players.
This is another side of grappling with the bitter lesson: What are the missing mechanisms of the human-AI economy? Some of them are standards, because standards and protocols are a big part of what makes an architecture of participation. I have a group working on portable memory, so that what your AI knows about you can move with you from one tool to another.
That story is just beginning. We are in an exploratory phase of mechanism design for AI, and it would be a mistake to settle on answers too early. Somebody is going to figure out the next mechanism, and we’ll all say, “Oh, that’s so damn obvious.”
What game are we playing? From architectures to incentives.
Mechanism design starts by asking what outcome you want. So what game ought we to be playing in the AI market?
We’ve been told that the game is the quest for superintelligence: a winner-takes-all contest that will bring untold riches to the winners and maybe impoverish, or even kill, everyone else. We’ve seen mechanisms designed to support that game. Massive investments in data center capacity mean that only the well-capitalized can play. There is colonialist-style appropriation of existing intellectual property. Access to models is sold through expensive APIs, and closed models come with terms of service that forbid others from learning from their outputs. Advertising may be creeping into answers, and memory locks your context inside one provider’s walls. Is that the game we ought to be playing?
I think the game we should be playing is human flourishing, and I think the game of human flourishing is creating new affordances for people to create and share with each other. We’re a social species. History teaches us that we will find our way there, but we can get there more quickly or more slowly depending on the choices we make. Getting there takes mechanisms that create a circulatory economy of value exchange between humans and their agents, with humans in partnership with AI rather than replaced by it.
The big labs, whatever the motives of the people who run them, are all driven by the imperatives of our financial system. They have raised enormous sums that they somehow have to pay back, so building a hugely successful business has to be their overriding goal, and that shapes all their incentives. In some ways AI was enshittified from the beginning, because it was built on the backs of so much capital that has to be repaid. That’s why open source AI matters so much, and sovereign AI too, built and controlled within individual countries. The open models, which anyone can download and run, aren’t as capable as the best closed models yet, but they keep getting closer, and they are already good enough for a lot of productive work.
Open source, open protocols, and open memory give users the freedom to shape and control the technology they use. The game I want us to play is one in which all of us build the future together, so that we control this wonderful new wave of technology for ourselves. I’m starting to see the shape of an economy where AI belongs to people rather than to the big labs, where we have personal AI agents working for us and with us, and we build an economy around that. I think it’s going to happen not because of the big labs, but in spite of them.









