Hayete Gallot, now executive vice president of Microsoft Security, speaks at a Microsoft event in France in 2024. (Microsoft Photo)
GeekWire is profiling over the next few weeks some of the people and teams that are shaping the evolution of Microsoft in what we’re calling its “Microsoft 2.5” era.
AI has had an impact on just about every tech-product category, but especially security. Attackers are using AI; customers are looking to defend with AI. The goalposts keep shifting. “Agentic security” is now the holy grail, and Hayete Gallot, the newly minted executive vice president of Microsoft Security, is leading the charge toward it.
Gallot, a 16-plus-year Microsoft veteran who rejoined the company in February after a 1.5-year Google detour, replaced Charlie Bell, who came to Microsoft from AWS in 2021 and continues at the company as an individual contributor focused on engineering quality.
“Customers care about two things: solving for security and being able to afford it,” Gallot said when I asked during our interview this week why she came back to Microsoft.
“I am a problem solver. And an engineer at heart (and by training). Security is the most important problem right now — and Microsoft is the only place with all of the puzzle pieces to help our customers.”
Since her return, Gallot hasn’t been shy about shaking things up. As noted recently by The Information, at least nine corporate vice presidents who previously reported to Bell have left the company this year.
“We’re making changes to ensure we’re in the best formation to go after this opportunity,” she acknowledged.
“I’m motivated by doing the right thing for our customers, my teams, and tech outcomes,” she said. “I like to move quickly: days and weeks, not months and years, learning through execution, iterating rapidly, and adjusting based on real customer signals.”
The company isn’t starting from scratch. As of 2021, Microsoft claimed security was a $10 billion business for the company. By 2023, security had reached a $20 billion annual revenue rate, officials said.
Those claims haven’t been without controversy. Microsoft has built a huge business in finding and fixing security problems which some customers felt were of the company’s own making.
Microsoft has a wide-ranging and rather unwieldy security portfolio, encompassing identity management (Entra), endpoint protection (Defender), endpoint management (Intune), security information and event management (Sentinel), and compliance (Purview), among others.
In 2023, Microsoft introduced its Security Copilot set of AI analysis services that integrated with some of its existing security offerings. But a portal-based solution like Security Copilot doesn’t offer the kind of end-to-end coverage that an agentic security platform can, Gallot said.
The problem is that attackers are using agents, too. Customers need real-time insight into what’s happening in their environment, and the ability to act just as quickly, Gallot said.
Agentic security is about “taking the signals and turning them into a graph that is useful,” Gallot said. “If you’re trying to reason about 100 trillion signals, it’s not really effective.” The graph, she said, lets agents pick the right model for each threat and close the loop.
In practice, that means the system can quarantine a device or revoke access on its own, for example, rather than waiting for a human.
Microsoft’s core existing security products will continue to play a role as the landscape evolves, both spotting the problems and acting on them. Security Copilot isn’t going away in the process: “You’ll have Copilot and you’ll have agentic security,” she said.
The company’s new Agent 365 “control plane” — a central console for tracking every AI agent a company runs — fits in by letting customers see the “blast radius” of an agent, meaning everything a hijacked agent could reach, Gallot said. It’s similar in concept to Zero Trust, the “never trust, always verify” security model that limited how far an attacker could get with a stolen employee login, but applied now to agents rather than people.
Traditional AI security and agentic AI security are fundamentally different, Microsoft says. Agentic security doesn’t just protect models and training data; it also can protect tools, workflows, memory, connected systems and more. Because agents can take action, the potential positive and negative stakes are higher.
While AI has helped businesses make strides in finding and fixing vulnerabilities, it hasn’t gone much beyond that. Microsoft introduced its multi-model agentic scanning harness (MDASH) as its first step into the agentic security space, Gallot said.
The company used MDASH internally to boost finding and fixing Windows security issues, and it is now making it available to select customers in an expanded preview. MDASH will allow customers to use the best model for the right task to secure all different types of code bases, she said.
Microsoft is rumored to be readying a more comprehensive agentic security offering, of which MDASH is likely just one piece.
Microsoft is far from the only one doing this. AWS, Anthropic, and OpenAI are offering security tools on their platforms, and dedicated security vendors are building their own agentic platforms.
Microsoft has the advantage of scale in the enterprise. The question is whether Gallot and her new leadership team can turn that scale and emerging AI tools into both a bigger business for the company and better protection for its customers.
Upgrades include automatically created documents and files, as well as interactive reports. Approvals and other repeatable work can be handled through automation.
Ryan welcomes VoidZero’s Evan You and Cloudflare’s Dane Knecht back to the show to discuss Cloudflare’s recent acquisition of VoidZero and what it means for JavaScript development, how partnerships like theirs can help open-source projects stay maintained and sustainably monetized, and how Cloudflare’s distributed systems are helping to improve developer experience in Vite and beyond.
“Help! There are too many [LLM bug reports, blog posts about LLM bug reports, books, treatises, codices, scrolls, papyri, cuneiform tablets]! How do I choose which to read?”
—Many people, presumably
Stop there! If you are reading this, ask yourself how you got here. Did Substack’s algorithm recommend this article for you? Did a juicy thumbnail provide a welcome distraction from a mundane task? Maybe you know me personally and feel you have an obligation (you do)? Are you already regretting your decision to click?
The maintainers of many of the most important open source software repositories in the world are “drowning” in bug reports.1 Daniel Stenberg, who runs curl, has documented a rising tide of such reports,2 generated in part by well-meaning users equipped with the latest LLMs. These reports look entirely plausible, and a minority of them actually highlight real vulnerabilities. But most are essentially worthless. Actually, they might be worse than worthless, since the only way to know whether a report reports something real is to do most of the work of validating it by hand. The cost of producing bug reports has diminished, while the cost of validating them has remained constant. Thus, this flood of LLM generated reports diverts expert maintainers who could be spending their time and attention on reports with a higher relative signal.
This is an instructive microcosm of a wider LLM-fueled dynamic. With the ascendance of LLMs, the cost of producing credible–looking work across many domains has plummeted. Recently, I prompted Claude Code to do some research on a relatively advanced idea I was mulling in the AI alignment space (representational similarity analysis over LLaMA activations for prompted deceptive intent detection). It spat out, in LaTeX, a whole paper, complete with data from experiments that it had actually run, p-values, equations, figures, a literature review, and a bibliography (which mostly included real papers). It should come as no surprise then that the submission volume to academic journals has risen 42% since the introduction of ChatGPT, while writing quality has declined.3 Indeed, my paper was pretty bad (no doubt in part because of the quality of the idea I gave to it), but it looked very credible and cost me almost nothing to produce. I think it would have taken a domain expert around 2–3 minutes to work out that it was slop, and quite a bit longer to describe its main flaws in detail.
This time cost will surely rise.
The cost of producing credible-looking papers, credible-looking cover letters, credible-looking code, credible-looking blog posts, credible-looking bug reports, credible-looking mathematical proofs, and credible-looking risk analyses is heading to 0. So the supply will continue to skyrocket.
In essence, we are now great at generating stuff, but much less great at figuring out whether that stuff is actually any good.
I am battling with this problem even as I write this. I use Claude to help me editorialize and think through my ideas—relatively little shame in that. But as I navigate Claude’s outputs, I am spending a lot of my time not really ‘collaborating’ but trying to work out which of the “strengths” of my writing that it has picked out are merely sycophantic rehearsals of my ideas, and which of the “weaknesses” highlight genuine flaws.
Here, I argue that credibility cost collapses have historical precedent. I suggest that when they occur, we tend to invent new sociotechnical gating mechanisms/institutions that help us work out how to allocate our attention. I then talk about what the gating mechanism for credible slop might look like, and what it should avoid.
Hidden gates, cost collapse, and credibility signaling institutions
When things are hard to make, the mere existence of the thing is evidence that someone has invested a great deal of time and money (which hopefully correlates with relevant expertise) into creating it, and thus it is likely credible and worthy of one’s attention. For several centuries before Gutenberg, making one book took a scribe a full year and a herd of animals’ worth of skin to make. Then, you needed a patron in order to buy one, and to read the thing you needed to know Latin.
When books were scarce, nobody took time to wonder whether one was worth their attention. Scarcity was the gate. Of course, a “scarcity gate” does not guarantee credibility—it is an imperfect filter. Furthermore, scarcity often brings with it the politics of access which restricts the ability to participate in the production and dissemination of information. Ideally, a thing would be scarce purely because one requires expert skill and knowledge to produce it—but, as in the book case above, this is often confounded by wealth, social circumstances, or access to education.
But then the cost of producing things decreases. The printing press replaces the scribe; cheap paper replaces vellum; literacy spreads; things start being written in modern rather than ancient languages; computer science becomes the most popular undergraduate degree. The playing field is leveled, and leveled in a powerfully democratic way; socioeconomic barriers to production and consumption of information fall away.
With this newfound abundance, the scarcity gate stops working and so comes the need for new ways to work out what is actually worth our attention. New socio-institutional gates have to be built. The classic example is the journal: For a century and a half after the arrival of Gutenberg’s press there was a major concern among intellectuals at the newfound surplus of available printed-word documents. Conrad Gessner, in 1545, in the preface of his Bibliotheca universalis lamented the “confusing and harmful abundance of books.” Barnaby Rich, a writer and sea captain, grumbled in 1613 that “one of the diseases of this age is the multiplicity of books.” The historian Ann Blair called this the problem of “too much to know,” the sense that there were now more books than anyone could read in a lifetime and no obvious way to tell the worthwhile from the dross (Too Much to Know, 2010).
Later, in the 19th century with the birth of industrialized printing, we got yet more complaints. See the following quote from Schopenhauer on “the immense number of bad books” available at the time:
…these rank weeds of literature, which deprive the wheat of nourishment and choke it. Thus they use up all the time, money, and attention of the public which by right belong to good books and their noble aims, while they themselves are written merely for the purpose of bringing in money or for procuring posts and positions. They are, therefore, not merely useless but positively harmful.4
Back in the 17th century the socio-institutional solution of curated journals emerged to save the day. In the space of two months in 1665, Denis de Sallo launched the Journal des sçavans in Paris and Henry Oldenburg launched the Philosophical Transactions of the Royal Society in London. What made these important was not that they stored knowledge but that someone now stood at the door and decided what got through it. Oldenburg solicited, selected, and vouched for, so that appearing in it was itself a signal. It was no longer costly to write, but it was costly to get one’s writing past Oldenburg and into the journal. Readers of the journal, insofar as they trusted Oldenburg’s judgment, were then confident of the quality of the material to which they were allocating their attention.
This is one type of gate, but we have created many more—we peer review, we certify speakers with degrees, we count how often they cite each other, we invite people whose work we know and/or like to speak at events, we check follower counts, we count how often websites reference each other, etc. We know these proxies are imperfect (see Didier Raoult’s h-index) but we use them because we need some way of deciding who/what to pay attention to.
AI is a truly novel technology in its radical generality, and thus one should certainly take care in reaching for historical analogies. But, insofar as today’s models can be understood as dropping the cost of producing credible looking media, I think it is helpful to think about how we have dealt with such circumstances previously. The appearance of credibility has been severed from real credibility many times, precisely when it is no longer costly to look credible, and (admittedly sometimes after a period of chaos and strife) the response tends to be to build an institution to make that appearance expensive again.
The question then becomes what the next gate(s) might possibly look like. When it costs nothing to produce credible-looking work across most disciplines, what can remain expensive and be charged for that is a satisfactory proxy for something worth our time? I think there are more good bug reports, good blog posts, and good web apps being developed now than ever before, but the issue is that there are also vastly more bad ones—we need a mechanism for telling them apart.
How to not throw the baby out with the bath slop
So what do we do? Previously, proxies were invented to figure out whether something was worth one’s scarce time and attention, prior to consumption.
The digital approach has, thus far, been to use popularity-contest style proxies. PageRank, Google’s original algorithm, used the number of other web pages that point at a given web page to rank their relevancy. Similarly, many of the recommendation algorithms you use daily, from Substack to Amazon, rely heavily on what people are currently viewing, engaging with, and buying. In other words, we allocate people’s attention to things that other people are already attending to. But the logic of these measures, like the ones discussed above, have a perverse feature: They do not really tell us whether something is worth our attention. Instead, they tell us how much attention this thing has already received, and we treat the second as a proxy for the first. Thus, your attention becomes both the input into the mechanism and the output. Whether or not this blog post appears in your feed is a function of how many people have clicked it before, so attention accrues attention, creating a classic winner-take-all type dynamic. Worse, the moment you have a sorting infrastructure whose currency is attention, the platform that owns the infrastructure has the proxy (engagement, ad revenue etc.) as the incentive and not the target (providing content that is worth people’s time). This is a dynamic that Tim O’Reilly, Ilan Strauss, and I have studied before in our work on algorithmic attention rents.5
The point is that AI did not break a working gate. In fact, in some ways, AI has helped; I have talked elsewhere about how ad-free LLMs are currently better search tools than many traditional search engines.6
In the context of credible-looking-slop though, AI is a dam buster. Domains that were previously reliant on human-judgment-based gating such as academic journals, open source software repositories, are getting flooded. And attention-algorithmic digital search and recommendation platforms are sagging under the combination of the slop strain and their own feedback loops. How many distinctly AI-y articles have you clicked on lately on Substack? I clicked into YouTube’s “shorts” on a logged-out computer the other day and was staggered by the unbridled slop it served up. If you, like me, have been forced to engage with LinkedIn’s feed since ChatGPT’s ascendancy late 2022, I offer you my sincerest condolences.
One candidate solution is that we lean harder on the human-centric institutional gates that we already have: reputations, followings, h-indexes, knowing someone who organizes really cool unconferences, etc. This certainly feels like the most likely direction of travel. However, it carries the cost of entrenching incumbents: Your papers only get read if you are at Harvard; your open source contributions only get accepted if you are already well known in the community; your blog posts only get seen if you are featured by someone with a platform. Central to the appeal of cheaper production is the democratization of contribution—if you are smart and have a good idea for an app or for some alignment research, you can get Claude to help you prototype it without having to learn the entire modern internet stack. The issue is that if genuinely good ideas never get seen because the only stuff people think is worth their time comes with a recognizable affiliation, we destroy that democratization. The baby goes out with the slop.
The second obvious candidate solution is to call for more AI. Every gate thus far has been a proxy—scarcity, the credential, the citation, etc.—that doesn’t directly measure the quality of the content. Rather, it measures something easier to capture that, hopefully, correlates with the quality of the content. What a LLM-based gating system seems to offer, for the first time, is a gate that can actually “read” all the content. One could envision a future where we all encode our preferences in personal-reviewer type models, which then actually go through the films, books and journal articles we are selecting from in order to provide personalized, reliable recommendations. The signal, in such a world, comes home to the object and stays cheap.
Unfortunately, this response seems to miss two important points. The first is a turtles-all-the-way-down problem: The gate and the thing it gates are drawn from the same well. The second is a problem of incentives.
A detector built out of frontier model capabilities may always inherit frontier model blind spots. If AI is capable of convincing itself that the slop it’s generating is the baby, then, if they are the same models, it may be enough to convince the reviewer too. Of course, it is not that LLMs can only ever emit credible looking content—they conduct real mathematics,7 write real code, submit real bug reports. But these are currently few of the total cases (the baby) among a lot of false positives. AI will get better, and eventually perhaps all of the bug reports it submits will be real, all of the proofs it generates will be correct, etc. This problem might dissolve as the systems get more intelligent. But we don’t know when/if AI systems will get to this point, and even when/if they do, presumably it will be quite a bit after that point before we trust them with doing all the stuff—building our planes, creating our medications, designing our policies, etc.
The second thing this response misses is incentives: What happens if we have two such super intelligent machines aimed at deceiving each other? Will an employer’s verification AI be able to see through the ruse of the applicant’s application AI? What about a deviant academic, who sets his AI to work writing a paper optimized for receiving citations? Will the journal’s editorial AI’s be able to catch subtle massaging of data or p-hacking?
We have developed truly sci-fi technology for generating content, but our infrastructure for evaluating its outputs, for curating them, and generally for exercising taste at scale has lagged behind. Maybe the answer lies somewhere between the two avenues I’ve suggested thus far. We have LLM reviewers filter the bug reports, perform some diagnostics, before passing to the human maintainers. But even this risks the identification problems I discussed above.
So I don’t have a clean gate idea to sell you on, I wish I did. Maybe ask Claude?
In September 2025, an attacker phished the credentials of a single npm maintainer and published booby-trapped versions of chalk, debug, and around a dozen other packages that are together downloaded more than 2 billion times a week. The code rewrote cryptocurrency wallet addresses inside any browser app that loaded it. The poisoned versions were live for roughly two hours before the community caught them and npm pulled them.
Two hours is a fast response. However, it is also more than enough time for an automated update tool to see the new version, open a pull request, and put it in front of your team, because version update tooling is built to grab the newest release the moment it lands.
That pattern sits behind a growing share of supply chain attacks. The malicious code rides in on a brand-new release, is published to a public registry, and gets pulled into build pipelines within minutes, before a human or a scanner has even looked at it.
A cooldown changes that math. Waiting a few days before adopting a new release gives maintainers, security researchers, and automated scanners time to spot a malicious version and get it pulled before it ever reaches your pull requests.
For non-security version bumps, Dependabot now waits at least three days after a release is published before opening a pull request. The cooldown configuration option in the dependabot.yml still controls the behavior, though, so you can choose a different cooldown parameter that fits your project.
Case studies and GitHub Advisory Database data
When attackers compromise a popular package, the poisoned version tends to have a short lifespan. It gets published, spreads through whatever installs it, and gets caught, usually within hours. The previous example was live for only two hours. Other widely used packages have followed the same arc, with compromised builds of Solana web3.js, Axios, and ua-parser-js each caught within a few hours of publication.
More generally, GitHub sees this pattern directly through the GitHub Advisory Database, which catalogs open source security advisories across ecosystems. In the year ending May 2026, the database published more than 6,500 npm malware advisories, up from roughly 6,200 the year before, which adds up to approximately 18 newly cataloged malicious npm packages every day. A cooldown keeps you out of that opening window and lets a release accumulate some scrutiny before it reaches you.
Why three days
Published malware targeting popular packages tends to get caught fast. A review of 21 widely reported supply chain incidents between 2018 and 2026 found the same pattern: malicious versions of axios, Solana web3.js, ua-parser-js, and Ledger Connect Kit were each pulled within hours of publication, and a cooldown could have filtered out the majority of these short-lived publishes before anyone installed them.
Three days as the default balances two goals: it pushes you past the window where most of these attacks live, and it doesn’t hold your dependencies back longer than necessary.
Other community members have also landed on a three-day cooldown (though some go even longer), so this default behavior keeps Dependabot consistent as developers move between tools.
You can always set a longer or shorter window with Dependabot’s cooldown configuration option.
Defense in depth
A cooldown is built for a specific pattern: a malicious version that ships, spreads, and gets caught quickly. It does little against attacks that play a longer game, including backdoors planted in releases and left dormant, maintainer sabotage, or a compromised build system. The point of the default is to remove a common and time-sensitive path, not to stand in for the rest of your defenses.
Because a cooldown only addresses the fast-moving case, it should be one layer among several. Some additional steps to take include pinning dependencies with lockfiles, disabling install scripts in CI where you can, scoping the tokens in your build pipelines, and reviewing updates before they merge.
This is one step among several we are taking to harden the software supply chain for everyone who builds on GitHub. It’s on by default, so you don’t have to change anything to activate it. You can also tune it to fit your workflow.
The first expensive agent run doesn’t look like a governance problem. It looks like a billing problem.
A team opens its first agent invoice after the meter turns on, sorts the runs by cost, and finds one that cost 40 times the median. The provider meter shows tokens and a total. The application logs say the request succeeded. The trace viewer shows a tidy request and a tidy response. None of them explain why this run wandered while its neighbors finished cleanly.
In my previous Radar article, “The Subsidy Ended: What Tool-Using Agents Actually Cost,” I argued that usage-based billing didn’t make agents expensive; it made their existing costs visible. The bill didn’t get bigger. It just got honest, and an honest bill is one you can engineer against.
But visible isn’t the same as attributable. To attribute cost in a tool-using agent, you have to see inside the run that produced it. Once you build that visibility, you discover that cost is only where the trouble first becomes visible.
Cost spikes, unsafe delegation, and runaway actions are different failures, but they expose the same missing layer: a control plane can’t govern a loop it can’t independently observe.
The bill is honest, but it isn’t explained
The number on the invoice isn’t wrong, only incomplete. Provider billing can tell you what was consumed; it usually can’t tell you which design choice inside your platform caused the consumption. Application logs can tell you whether the outer request succeeded; they often can’t tell you how the agent got there. That leaves teams arguing over a bill when the thing they need is an audit trail.
By control plane, I mean the platform layer above individual agents where an organization centralizes observability and enforces policy, access, budget, routing, and execution constraints. Most organizations have pieces of that layer already. What they often lack is the evidence layer underneath it: a loop-aware record of what the agent actually did, turn by turn.
The control plane is where policy decisions live. The observability substrate is the evidence the control plane reads from. The instrumentation points are the runtime chokepoints the agent can’t bypass: model gateways, tool proxies, API gateways, execution sandboxes, runtime harnesses, and policy engines.
Many organizations instrumented the application boundary, then deployed systems whose real work happens inside a loop. The result is a control plane with opinions but not enough evidence.
The loop is the unit of observation
Here’s the mistake underneath the empty trace. Agent observability is often treated as a heavier version of application observability, when it’s a different shape entirely. The unit of work changed, and the instrumentation didn’t. A traditional service handles a request and returns a response; the request is the natural unit you trace.
An agent doesn’t so much handle a request as work toward an outcome. It reasons, calls a tool, reads the result, reasons again, and continues until it decides it’s finished, hits a boundary, or escalates. A single user intent can fan out into many model calls, many tool calls, and a context window that changes on every turn. The signal that matters is the relationship between those turns, not only the timing of any one of them.
Figure 1. From request trace to loop trace. A request-response trace shows that something completed. A loop-aware trace shows why the agent took the path it took: which turns ran, what context accumulated, which tools were called, which controls fired, and what each turn cost.
Three things follow from this, and each one breaks an assumption that application monitoring quietly depends on.
First, the context is accumulating state, not a fixed payload. Each turn may carry forward prior messages, tool descriptions, retrieved files, intermediate results, and earlier decisions. You have to be able to watch that state grow turn by turn, because the growth is where much of the cost and risk live.
Second, a tool call is a first-class decision, not an implementation detail. Which tool the model selected, what parameters it passed, how large the result was, and whether a policy constrained the call are all part of the governance record. Routing accuracy and routing cost are the same audit viewed from two directions.
Third, every run can become its own trace tree. The same prompt can take a different path on Tuesday than it took on Monday, so fixed call graphs and clean service maps assume a regularity the agent may not have. If the unit of observation is still the request, you will see 10,000 successful calls and never notice the one loop that ran 15 turns when it should have run three.
What the substrate has to capture
Once you accept that the loop is the unit, the requirement becomes concrete. You need a small, specific set of signals captured below the agent and stored where you can query across the whole fleet, not only inside a per-run viewer. In a pilot I’m running for a large healthcare organization, this is the layer we built first, on OpenTelemetry, Cloud Trace, and a usage-log table in the warehouse. The particular stack matters less than the shape, which generalizes well beyond it.
Figure 2. The observability substrate. Instrumented at the layer every model call and tool call must pass through, the same signals land in a fleet-queryable store and answer governance questions about cost, delegation, and runaway actions.
At minimum, each user intent should produce a run trace. Each loop turn should be represented as either a span or a stable grouping attribute. Model calls, tool executions, policy checks, retries, and postprocessing should be child spans or structured events beneath that turn. The exact naming convention isn’t as important as preserving the causal structure of the loop.
Signal
Why the control plane needs it
Example fields
Run and turn structure
Keeps the run legible as a causal tree rather than a flat list of calls
run_id, turn_id, parent_span_id, timestamp
Token and model accounting
Makes cost explainable per turn, model, and tool path rather than merely visible in aggregate
model, input_tokens, output_tokens, cached_tokens
Tool-call events
Records delegation decisions and identifies oversized or repeated tool results
None of this is exotic, and the practical design work isn’t inventing new telemetry primitives but controlling cardinality, retention, payload capture, sampling policy, schema evolution, and the joins between trace data, usage data, identity data, and policy data.
The storage point is the part teams underestimate. If these signals land only in a tracing viewer, you can inspect one run beautifully and never reason about a thousand. Governance is a fleet question, not a single-trace question, so the substrate has to be queryable.
It also has to be designed with data minimization in mind: metadata by default, content capture by exception. Capturing a tool call doesn’t mean storing every raw prompt, full result set, credential, confidential document, or sensitive parameter in the trace. In regulated environments, the useful pattern is to separate metadata from payload: tool name, model, token counts, payload size, row counts, policy decision, authority context, request ID, and redacted or hashed parameter values where necessary. The goal is enough evidence to reconstruct why a run behaved the way it did, not an uncontrolled archive of everything the agent saw.
The first useful version doesn’t need full prompt capture or semantic evaluation. With columns like run_id, turn_id, parent_span_id, timestamp, principal_id, delegated_scope, model, input_tokens, output_tokens, cached_tokens, tool_name, result_bytes, row_count, policy_id, policy_decision, stop_reason, loop_bound_hit, and outcome_status, expensive loops stop being mysteries and start being queries.
The exact syntax will vary by warehouse, but the governance question should be expressible without a human clicking through individual trace viewers:
with runs as (
select
run_id,
count(distinct turn_id) as turns,
sum(input_tokens + output_tokens) as total_tokens,
max(result_bytes) as largest_tool_result,
bool_or(loop_bound_hit) as hit_loop_bound,
count_if(policy_decision = 'rewrite') as rewritten_actions
from agent_turn_events
where occurred_at >= current_date - interval '7 days'
group by run_id
)
select *
from runs
where turns > 10
or largest_tool_result > 10000000
or hit_loop_bound
or rewritten_actions > 0;
That is the difference between admiring a trace and governing a fleet.
In the old trace, the expensive run from the opening was simply expensive. In the loop-aware trace, it becomes legible: turn 3 retrieved 80,000 rows, turn 4 carried that result forward, turn 5 selected the expensive model, turns 6 through 11 retried the same tool call with slightly different parameters, and the run finally stopped because it hit a loop bound rather than because it completed cleanly. The run stops being a riddle and becomes a record.
One substrate, three governance problems
The reason this is worth building once, properly, is that the same substrate answers the three agent governance problems that the industry often treats as separate: cost management, delegation and access control, and runaway-action prevention. They are not identical failures, but they require the same kind of evidence.
Governance problem
Evidence the control plane needs
Cost
Turn count, token counts, model selection, context growth, tool-result size, retries, and stop reason
Delegation
Principal, delegated authority, data scope, selected tool, action parameters, and policy decision
Runaway actions
Repeated actions, loop bounds, payload caps, guardrail decisions, denied or rewritten actions, and outcome status
Cost is the first, and with token accounting on every turn you can finally answer why a run was expensive. You can see whether the cost came from too many turns, too much context carried forward, an oversized tool result, an expensive model used for the wrong step, or a retry loop that should have been bounded.
Delegation and access are the second, and harder, problem. In multi-agent systems, delegation is a security boundary. Enterprises will eventually be asked who authorized a given agent action, under whose authority it ran, and which data scope applied at the time. The audit trail for that question is this same trace, enriched with identity and authority on each turn.
Runaway actions are the third. The destructive delete that becomes a war story, the agent that tried to drop a production table, or the loop that repeatedly issued the same expensive scan shouldn’t only exist in a postmortem. In this model, the blocked destructive statement is a guardrail decision event with a deny on it, and the runaway scan is a trace that hit a loop bound or payload cap. The interesting governance signal is the dangerous action that a deterministic control refused.
Three conversations, one place to stand. The loop is the unit of governance because the loop is where cost accumulates, authority is exercised, tools are selected, controls fire, and outcomes emerge.
The agent can’t keep its own records
There’s a tempting shortcut to instrument the agent itself, to let the agent log its own tokens, its own authority, and its own blocked actions. That’s the fox keeping the henhouse ledger.
The agent can emit useful breadcrumbs, but it can’t be the system of record for its own authority, cost, or refusals. An agent reporting on its own scope and blocked actions is self-reporting, and self-reporting is exactly what fails an auditor and exactly what a clever prompt can talk its way around.
The substrate has to be instrumented below the agent, at the layer the agent can’t opt out of. In practice, below the agent means the model gateway, tool proxy, runtime harness, execution environment, API gateway, or policy engine: the layer the agent has to pass through, not a logger the agent can choose to call.
This is the through-line of the control-plane argument. The platform is where you enforce policy, access, budget, routing, and cost, and it can only enforce what it independently observed. Enforcement and observation are two faces of the same layer; put them anywhere the agent can edit, and you have neither.
We already have tracing, and it isn’t enough
The natural objection is that this is solved already: Mature tracing tools exist, agent observability vendors exist, and teams can turn on a trace viewer and see what happened. The gap isn’t visualization, since plenty of tools can show a useful trace of an agent run. The harder gap to cross is completeness and actionability: whether the trace carries the evidence a control plane needs, whether that evidence is independent of the agent, and whether it lands somewhere the organization can query across the fleet.
Existing layer
What it often shows
What the control plane still needs
Application tracing
Request, service call, latency, status
Turn structure, context growth, model and tool attribution
Agent run viewer
One run’s path through a UI
Fleet-queryable evidence across all runs
Agent self-logging
Model-reported actions and reasons
An independent record below the agent
Billing dashboard
Total cost and token usage
Per-turn causal explanation of where the cost came from
A useful test is whether the control plane can answer this without opening an individual trace viewer: Show me all runs this week where context grew by more than 5x, a tool returned more than 10 MB, a guardrail rewrote the action, and the run still reached a user-visible answer. If the answer requires a human clicking through traces one by one, you have visualization, not governance, and seeing one run isn’t the same as governing a thousand.
A dashboard tells you what happened. A control plane uses what happened to change what happens next, which requires the signal to live somewhere an enforcement decision can read it.
The pattern, not the stack
It would be a mistake to read this as an argument for a particular tracing standard, warehouse, vendor, or cloud platform. The stack is incidental; the shape is the point.
The recipe stays the same regardless: loop-aware traces; turns represented as spans, grouping attributes, or structured events; token, tool, guardrail, and identity evidence attached to those turns; storage you can query across the fleet; instrumentation that sits below the agent rather than inside it; and data minimization that keeps the trace useful without turning it into a shadow copy of sensitive payloads. Build it on whatever your platform already speaks.
The teams that treat observability as a dashboard will keep discovering their problems in the order the symptoms happen to surface: first as a surprising invoice, later as an audit finding, eventually as an incident. The teams that treat observability as the sensory layer of the control plane will see all three coming from the same data, and will be able to act before the meter, the auditor, or the incident forces the question.
Prompts guide behavior. Guardrails govern behavior. Observability is how you know the governance is real. You can’t govern what you can’t see, and you can’t improve what you can’t attribute.
Log in
︎
