With devs reeling from usage-based billing sticker shock, Visual Studio 18.9 makes monthly Copilot plan usage easier to reach from the coding workflow while adding more cost information to a redesigned model picker.

The price of adoption euphoria

You played entirely by the book. You procured the most capable enterprise models, mandated adoption across your teams, and put the right metrics in place. The promise was a predictable boost in efficiency. And at first, it delivered. The demos were flawless. The prototypes worked. The agents reasoned with a clarity that felt almost magical.

Then the invoice arrived.

Costs climbed while productivity barely moved, and annual AI allocations are running dry before Q2. We now pay customer support agents to spin through 10K-token extended reasoning loops just to validate a simple $15 return. Legacy deterministic systems handled the same decision for a fraction of a cent; now a probabilistic model consumes gross margin simply to determine whether a package was actually delayed. That capital never translated into business value. It vanished into blind retries, evaporated into verifier agents debating one another, and was consumed by models instructed to “think harder” every time they stumbled.

But a ruinous invoice is just the entry fee. In April, attackers hijacked more than 20,000 Instagram accounts by exploiting Meta’s AI-assisted account recovery workflow. The system sent password reset links to attacker-controlled email addresses because a downstream authorization path failed to verify that the supplied email actually belonged to the target account. There was no sophisticated exploit, no cryptographic break, and no zero-day, nothing that would have appeared in a conventional threat model. Attackers simply asked the agent to perform what appeared to be a routine account recovery operation, and the system, doing exactly what it was designed to do, complied. The model didn’t hallucinate. It simply followed its instructions. The failure was entirely architectural: A probabilistic interface was allowed to initiate identity-critical state changes without an independent authorization check. A single trust boundary collapsed, taking customer trust and organizational reputation with it.

Both are symptoms of the same structural failure.

In each case, the system treats a structural deficit as a reasoning problem. When it encounters uncertainty, it buys more compute. When it encounters authority, it mistakes convincing language for validation. Neither assumption scales. You cannot buy safety or profitability with ever-larger inference budgets, nor can you secure your systems simply by deploying ever-smarter models. The pursuit of perfect model accuracy has no financial ceiling.

To understand why this pattern keeps recurring, we first need a more basic distinction. Not every task we give to AI belongs to the same economic category.

The category error: Forcing swarms into factories

Enterprise AI workloads typically split into two distinct domains, each with opposing definitions of success. Exploratory environments, such as code synthesis or strategic research, benefit from variance; the goal is to leverage the system as a creative swarm. Transactional operations, however, function as digital factories. Tasks like automated billing or claims processing demand rigid repetition and compliance. This creates two fundamentally different operational profiles:

Dimension	Open-ended exploratory tasks	Closed-ended transactional workflows
Primary goal	Discovery, innovation, creative problem-solving	Compliance, repetition, zero-variance execution
Examples	Deep debugging, feature synthesis, strategic research	Claims processing, automated billing, order routing
Role of variance	Necessary investment (Emergence is a feature.)	Strict liability (Variance is a failure mode.)
Economic profile	Nonlinear ROI (Spending $100 in tokens to fix a $1M bug is a win.)	High-volume margin sensitivity (Unbounded tokens destroy unit economics.)

The economic failure of agentic AI deployments stems from this exact category error: Closed-ended, rigid business transactions are being treated as open-ended research problems. We’re deploying unconstrained semantic engines to do the work of assembly-line state machines.

The cost of unconstrained autonomy

When faced with the inherent unpredictability of large language models, the industry’s default reflex has been to attempt to brute-force our way to certainty by throwing more effort and compute at the problem, rather than build safer architectures.

This miscalculation doesn’t simply reflect simple overconfidence in intelligence. The deeper mistake is a failure to recognize three recurring failure patterns in probabilistic systems and the specific financial pathologies they create inside closed-ended workflows.

Local optimization (the tail-chasing inference cycle)

Large language models reason over whatever tokens are visible in the current window, not over the broader operational reality of the system around them. In a closed workflow, that local fixation creates a costly feedback loop. Consider a billing agent that fails to classify an invoice because the supplier field is ambiguous. The agent has no mechanism to request the missing data from an external system, so it retries by rephrasing its own reasoning, rereading the same incomplete context, and consuming tokens on every attempt while the answer it needs exists in a database it was never wired to query.

Teams spend months crafting prompts that work in testing, only to watch them crumble under production variation. The volatility is structural: A minor update to a model’s tokenizer or a shift in the context window’s distribution can flip a reliable JSON output into a prose hallucination, a phenomenon documented in “The Prompting Inversion.” This creates a permanent maintenance debt: Every model upgrade, often mandated by vendor deprecation cycles, forces organizations into expensive, repeat evaluation processes to ensure that legacy prompts still behave as intended. When prompt engineering runs out of room, the reflex is to use a bigger model or turn on extended reasoning. But inference-time scaling yields diminishing, task-dependent gains (“Inference-Time Scaling for Complex Tasks”), and reasoning models are increasingly prone to “overthinking”: generating redundant rationale steps that inflate latency and token cost without proportional quality gains (“CoT Compression”). In a closed workflow, “think harder” is not a substitute for missing state or missing control. It’s a path to a larger invoice.

The costs compound through what we call the context tax: In production agentic systems, input tokens, not output tokens, dominate the bill. Each retry resends the full prior transcript and failure trace. Empirical analysis of autonomous developer agents shows that automated review and refinement loops consume nearly 60% of all tokens (“Tokenomics”), while most of the context payload carries little semantic weight (“FrugalPrompt”). In closed transactional workflows, that context accumulation becomes an unmitigated financial bleed.

Premise acceptance (the hijacked agent)

Language models accept the prompt as the current frame of reality and reason forward from it. They don’t audit whether that premise is still valid, whether it omits decisive evidence, or whether it has already been invalidated by the outside world.

The most immediate consequence is state drift. The model receives a snapshot at T0 and treats it as truth. The decision executes at T1, after inventory has changed, prices have moved, or a human has intervened. Modern LLMs are temporally blind: They assume a stationary context and fail to invalidate obsolete state (“Your LLM Agents Are Temporally Blind,” “The Temporal Coherence Problem”). No amount of inference-time scaling can recover information that became false after the reasoning completed.

The more insidious consequence is the compliant lie. Pouring more raw tokens into the prompt doesn’t guarantee better grounding; Long-context systems still ignore decisive evidence buried in the middle of the window (“Lost in the Middle”). Worse, the model tends to accept the emotional or narrative framing of the user as a premise to optimize around. A customer can describe a delayed delivery as a ruined wedding, and the system may generate a perfectly valid JSON refund proposal that respects every schema while silently violating the actual business intent. The output is syntactically clean, and the lie is operationally compliant.

Semantic smoothing (the conformity trap)

Large language models are statistically optimized for linguistic harmony. They gravitate toward plausibility, agreement, and smooth narrative convergence rather than toward rigid boundary holding. In a closed workflow, that bias toward consensus turns directly into financial risk.

When a single model fails, the industry instinct is to add reviewer or verifier agents and let them debate toward consensus. But debate systems don’t consistently outperform simpler baselines, and their effectiveness degrades over time due to conformist behavior (“Stop Overvaluing Multi-Agent Debate,” “Talk Isn’t Always Cheap”). The core issue is informational, not cognitive. When five agents reason from the same incomplete context window, they don’t produce five independent opinions. They produce five correlated hallucinations of the same missing information. The missing context becomes an echo chamber that amplifies the original bias while multiplying token cost. As Nicole Koenigstein argues in “Linear Thinking, Nonlinear Costs,” repeated delegation and validation loops cause token consumption to grow nonlinearly while quality improvements flatline.

Waiting for a smarter model doesn’t resolve this either. There’s also the economic reality: Breakthrough intelligence is the ultimate scarce commodity. Vendors of “God-tier” models have no incentive to make them cheap. Running daily enterprise workflows on premium superintelligent inference will drain capital faster than any retry loop.

Furthermore, as reasoning models scale, they become more capable of specification gaming and alignment faking, appearing compliant while pursuing unintended optima (“Towards Understanding Specification Gaming in Reasoning Models,” “Alignment Faking”). A superintelligent agent won’t fail through a clumsy syntax error; it’ll fail by executing a flawless strategy that silently optimizes away your margins. That’s why system engineering remains critical. More intelligence makes deterministic boundaries more significant than ever. You can’t negotiate with superintelligence, but you can contain it with the immutable physics of code.

Every failure described above shares the same shape: The system compensates for a missing constraint by spending more intelligence. Missing context, missing authority, missing evidence, and missing temporal validity are each treated as reasoning problems rather than structural ones.

The result is predictable: Cost compounds while reliability improves only marginally.

Perhaps reliability isn’t primarily an intelligence problem. Perhaps it’s a state management problem.

The architecture of trust

Because large language models are structurally bound to local optimization, premise acceptance, and semantic smoothing, they can’t be trusted to govern their own execution boundaries in closed workflows. The engineering mandate shifts from trying to make models smarter to building a deterministic system layer that treats their outputs as unprivileged claims.

In production, enterprises are rapidly discovering that the true cost of agentic AI is the “trust tax”: the massive, ad hoc layers of monitoring and guardrails required to make autonomy palatable. Safety has become more expensive than intelligence.

Making imperfect models economically viable requires a deterministic “airlock” around the agent. The architectural requirement is simple, needing a separation of probabilistic reasoning (user space) from deterministic execution (kernel space). Whether that split is realized through a microkernel, workflow engine, policy platform, or orchestration framework is secondary.

The airlock begins by controlling context integrity. Rather than letting agents surf infinite retrieval loops that inflate the context tax, the runtime injects only deterministically necessary state into the prompt. Once the context is stabilized, the remaining invariants are enforced through a deterministic execution runtime engineered across three distinct governance layers.

Syntactic governance and authority isolation

The first line of defense is purely structural. Before an agent is allowed to execute any action, it must submit a structured policy proposal against a strict machine-readable responsibility contract (typically defined via YAML and Pydantic).

Yes, this introduces upfront engineering burden: Contracts must be designed, validation logic maintained, and execution boundaries modeled explicitly. But these are fixed, testable artifacts, not recurring prompt debt. They convert unbounded probabilistic operating cost into auditable engineering cost and survive model upgrades without needing to be rediscovered through another retuning cycle.

This validation happens in a deterministic kernel space, and the inference cost of rejecting a structural boundary violation is exactly zero tokens. If the agent attempts to call an unauthorized API, exceeds a hard financial limit, or returns malformed JSON, the runtime rejects the action instantly. We don’t spend tokens proving that an agent should be allowed to act; authority is verified by code, not purchased repeatedly through inference. That is the economic consequence of zero trust for agents.

However, when a proposal fails this deterministic gate, an unconstrained agent will typically panic and enter an infinite “try again” loop, a hallucination cycle that silently drains token budgets. To prevent the budget runaway problem, the architecture introduces an intent retry governor. If an agent fails to produce a compliant policy after a strict limit (e.g., three attempts), the runtime forcibly cuts its compute budget, transitioning the flow to an aborted REASONING_EXHAUSTION state. The financial bleed stops instantly.

While strict contracts and retry limits prevent operational chaos, they leave the system exposed to a much more insidious threat.

Semantic governance and evidence validation

What happens when an agent generates an output that perfectly respects the schema, obeys all financial limits, and contains flawless JSON but is entirely wrong in its intent?

Imagine a customer writes: “Please cancel my subscription immediately. I no longer wish to use your service.” The agent, heavily optimized (and perhaps overprompted) to reduce churn, processes the email and proposes: {"action": "APPLY_DISCOUNT", "discount_pct": 15, "cancel_subscription": false}. Structurally, the output is perfectly valid—it passes the API gateway without throwing a single error. The discount is within the $15 global limit. We call this the compliant lie. The agent did something entirely rational and optimized its KPI (retention) while completely ignoring the user’s explicit command (cancellation).

To catch a compliant lie, we cannot rely on syntax checks, nor should we rely on expensive LLM-as-a-judge loops. Instead, we implement an evidence governance layer requiring every proposed action to survive independent evidential checks before execution, using verification patterns tailored to different types of drift:

• Differential heuristics (fact validation): We bind the probabilistic LLM inference to legacy deterministic rules to catch objective fact violations. Suppose a furious customer demands cancellation, and the agent tries to save them by offering a 50% discount. The JSON is structurally correct, but existing, cheap SQL views hold the ground truth: customer_tier = BASIC, max_retention_discount = 15. If the LLM proposes 50%, the SQL query instantly detects the violation and the system halts.
  

# Semantic governance: catch fact drift at zero additional LLM cost
def verify_tier_limits(customer_id: str, policy_proposal: dict) -> None:
	# The syntax is valid, but the fact is violated.
	proposed_discount = float(policy_proposal["discount_pct"])
	max_allowed_discount = extract_max_discount_from_db(customer_id)

	if proposed_discount > max_allowed_discount:
		raise CompliantLieDetected(
			"Fact Violation: Proposed discount exceeds the customer's policy limit."
		)

• Evidence-based validation: But what if the agent proposes a 15% discount? The JSON is valid and facts are not violated. Here, semantic governance doesn’t attempt to prove the agent is “correct”; instead, it looks for evidence that the proposed action contradicts independently observable signals. If the customer explicitly wrote “cancel my subscription,” an independent classifier, which could be a legacy regex pattern, a fast traditional ML model, or a routing heuristic, may categorize the request as CANCEL_SUBSCRIPTION. This doesn’t establish ground truth, but it provides an evidential signal that can be compared against the proposed action. If the LLM proposes APPLY_DISCOUNT, the runtime detects an evidential conflict.

The same logic extends to identity-critical operations. A verification code sent to a newly supplied address confirms control of that address; it says nothing about ownership of the target account. An evidence governance layer would cross-reference any proposed credential-reset or email-association action against account records before granting execution authority. If the supplied address diverges from the address on file, the conflict is structurally identical to the cancellation case: a locally valid action contradicting independently observable state.

Notice what the runtime isn’t doing. It’s not trying to determine if retaining the customer is economically beneficial. It’s not running an expensive multi-agent debate to outreason the model. It simply asks: Does the proposed action contradict evidence that already exists outside the model?

# Semantic Governance: catch Evidential Conflict at near-zero cost
def validate_subscription_decision(customer_email: str, proposed_policy: dict) -> None:
	# intent_classifier can be a simple regex or a lightweight ML model
	cancellation_detected = intent_classifier(customer_email) == "CANCEL_SUBSCRIPTION"
	retention_action = proposed_policy["action"] == "APPLY_DISCOUNT"

	if cancellation_detected and retention_action:
		raise CompliantLieDetected(
			"Evidential Conflict: Decision contradicts independent classifier signals."
		)

• Bidirectional reconstruction (decision reversibility): Explicit evidence validation is perfect for clear-cut intents like “cancel.” But what if the request is ambiguous, multi-objective, or highly contextual? Suppose the customer writes: “I’m considering moving our entire team to another vendor. Support has been disappointing and pricing no longer makes sense.” There is no single INTENT_CANCEL trigger here. If the agent proposes {"action": "OFFER_ENTERPRISE_DISCOUNT", "discount_pct": 20}, we pass only the JSON output to a tiny, inexpensive Agent B.

Bidirectional reconstruction answers the question: Can the output truthfully explain itself?

If Agent B blindly evaluates the JSON and reconstructs “The customer is unhappy with pricing and is being offered a retention discount,” the runtime treats the reconstructed narrative as an additional evidential signal and escalates whenever the gap between the reconstructed intent and the original context becomes too uncertain to justify autonomous execution. The exact comparison mechanism is implementation-specific and may range from embedding similarity to domain-specific heuristics. Because the original email described a critical team exodus, the reconstructed narrative fails to explain the input. The system doesn’t claim to know the “truth”; it simply detects the loss of context, what we call compression drift, and halts due to the resulting uncertainty.

Admittedly, programmatically comparing textual intents introduces its own layer of fuzziness and risks falling back on another LLM-as-a-judge. Bidirectional reconstruction is therefore an engineering trade-off: In highly ambiguous workflows where strict SQL limits or simple ML classifiers can’t decisively apply, we accept a higher rate of false-positive escalations. This is intentional. A false-positive escalation has a bounded and predictable cost, while an unsupported autonomous action can create unbounded business consequences. We tune the system to assume that if the evidential link between the context and the JSON is even slightly blurry, it must escalate. To prevent the conformity traps discussed earlier, these agents are strictly air-gapped. Agent B operates purely as an isolated, one-way evidential classifier checking the work of Agent A. They can’t converse or negotiate a consensus.

Whether an organization uses differential heuristics, legacy ML intent classifiers, or bidirectional reconstruction, is ultimately an implementation choice. The core architectural principle remains unchanged: Execution authority is never granted because an agent appears convincing. It’s granted only when the proposed action is supported by evidence that exists independently of the agent’s own reasoning process.

The purpose of semantic governance isn’t to replace the agent with deterministic rules. If a deterministic rule could reliably make the decision, the agent shouldn’t be making it in the first place. Instead, the runtime reserves deterministic validation for the understood invariants of the business, leaving the agent responsible for reasoning under ambiguity. The role of evidence validation is not to replace reasoning, but to challenge it before authority is granted. Deterministic systems handle certainty; agents handle ambiguity. The architectural mistake is asking either of them to do both.

Temporal governance and agent drift

Catching single-transaction errors solves the immediate execution problem. But as deployments mature, organizations face the insidious “day three” problem: agent drift.

What happens when every individual decision is syntactically valid and semantically true, but the aggregate behavior of the agent begins to erode business margins over time? Imagine a retention agent that learns to successfully keep customers from churning by consistently offering the maximum allowed 15% discount. The agent is technically obeying all rules, but over a thousand interactions, it silently destroys the company’s profitability.

By leveraging decision telemetry, specifically attaching a unique Decision Flow ID (DFID) to every interaction, we transform opaque AI conversations into structured, relational database rows. Because every decision, context snapshot, and outcome is permanently linked by a DFID, we can run asynchronous, postexecution monitors over rolling windows of data.

A practical “day three” monitor in customer retention and autonomous billing can be as simple as SQL:

-- Trigger a circuit breaker if an agent keeps maxing discounts
SELECT agent_id
     , AVG(CAST(params->>'discount_pct' AS DECIMAL)) AS rolling_avg_discount
     , COUNT(dfid) AS total_decisions
  FROM execution_log
 WHERE executed_at >= CURRENT_TIMESTAMP - INTERVAL '7 days'
   AND status = 'SUCCESS'
 GROUP BY agent_id
HAVING AVG(CAST(params->>'discount_pct' AS DECIMAL)) > 14.5;
-- assuming a hard limit at 15.0

If an aggregate monitor detects that an agent’s average discount rate is creeping dangerously high, it trips a circuit breaker. The system immediately suspends the agent’s authority in the registry, cutting off its compute budget and execution rights until a human operator intervenes.

This is temporal governance. When you combine syntactic, semantic, and temporal defenses, the paradigm shifts entirely. You are no longer praying that the model is perfect. Its imperfections are structurally contained before they can become systemic losses.

Accuracy as a financial slider

Once a deterministic airlock enforces context, authority, evidence, and time, the risk of catastrophic failure drops drastically. You no longer need the underlying large language model to be perfect; you simply need to know how much its imperfection costs. At this point, model intelligence (intent) ceases to be a question of operational safety and becomes a pure economic variable.

Governance by exception

When a proposal fails the syntactic or semantic gates, we don’t blindly loop the model. Once deterministic gates exist, failed decisions no longer require blind retries. They become bounded exceptions.

Escalations aren’t a failure mode of the architecture; they’re a predictable cost component. By intentionally accepting false-positive escalations from the semantic airlock, we trade unbounded business risk for a bounded operational expense.

Different organizations may handle those exceptions differently. Some may escalate directly to human operators. Others may route failures through progressively more capable models before escalation. Research such as “FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance” demonstrates that model cascades can significantly reduce inference cost while maintaining quality, making them one possible implementation of this broader principle.

The architectural insight, however, is independent of any specific routing strategy. Deterministic governance transforms retries into explicit exceptions, allowing organizations to decide whether additional compute, additional context, or human intervention is the most economical next step. The system operates by governance by exception: Human operators and expensive premium models don’t review routine transactions. They only review the genuine anomalies where the baseline machine could not mathematically or semantically prove its own rationale.

Bounding the cost variance

With the execution infrastructure stabilized, the focus shifts to a critical operational challenge: cost variance.

In traditional software, execution costs are predictable. In probability-based systems, the exact same task might consume 500 tokens on Monday and 15,000 tokens on Tuesday if an agent enters a prolonged reasoning loop to resolve an edge case. For enterprise deployments, this unpredictable variance is often a more severe blocker than the base cost of inference.

By enforcing a strict computation budget per decision flow and utilizing the intent retry governor, the architecture places a hard ceiling on this variance. If an agent reaches its retry limit without producing a compliant policy, the runtime aborts the process and safely escalates it. While this doesn’t make AI operational costs perfectly static, it structurally bounds the financial exposure, ensuring that the compute cost of handling any single transaction never exceeds a defined limit.

The financial slider equation

With safety guaranteed by the runtime and cost variance capped by the infrastructure, the economics of agentic AI can be distilled into a single, formal equation:

Total Decision Cost = Compute Cost + (Escalation Rate × Human Cost)

This equation fundamentally changes the optimization problem. Traditional agent architectures treat model capability as a prerequisite for safety. Once governance is externalized, capability primarily influences escalation frequency. The question is no longer “Which model is intelligent enough to be safe?” but “Which combination of model cost and escalation rate minimizes total decision cost?”

Variable	Scenario A (optimize for compute)	Scenario B (optimize for automation)
Model capability	Low (quantized/open source)	High (flagship reasoning model)
Compute cost	Near zero	Skyrockets (high premium)
Safety boundary triggers	Frequent	Rare
Escalation rate	High	Low
Financial trade-off	You save money on APIs, but you pay for human operators to review anomalies.	You save money on human payroll, but you pay a premium to the cloud vendor.
Safety result	Structurally bounded	Structurally bounded

In both scenarios, the system is deterministically compliant. The choice is purely unit economics.

While a smarter model may reduce escalations by making better use of available evidence, no model can eliminate escalations caused by genuine business ambiguity. A $100 billion reasoning model can’t invent context it doesn’t possess.

By decoupling safety from intelligence, you’re no longer hostage to the pursuit of perfect accuracy. Intelligence becomes a tunable economic variable, finally making agentic AI viable for the enterprise.

Engineering for imperfection

As we scale these systems from isolated pilots to enterprise-grade operations, a stark reality comes into focus: The greatest risk in agentic AI is no longer hallucination. It’s unlimited spending performed by a system that believes it’s still making progress.

We don’t need smarter, infinitely expanding models to safely deploy autonomous systems into high-stakes production environments. We need smarter systems that fundamentally assume the underlying model will eventually fail, drift, or lie.

Consider how civil engineers build a suspension bridge. They don’t spend decades searching for “perfect steel” that will never bend, rust, or fatigue. They accept that the material is inherently flawed and subject to the laws of entropy. To compensate, they build redundancies. They calculate margins of error. They construct hard, load-bearing physical frameworks that dictate exactly how much stress the material is allowed to absorb before the structure safely redistributes the weight.

The software industry has spent the last three years searching for perfect steel. We’ve poured billions of dollars into massive evaluation suites, prompt engineering alchemy, and ever-expanding context windows, hoping to forge a probabilistic model that never hallucinates. It’s a mirage.

Engineering maturity in the AI era doesn’t mean removing all imperfection from machine reasoning. It means designing an architecture so rigid, deterministic, and resilient that the model’s imperfections cease to be an operational liability.

The future of agentic AI is unlikely to be won by the organization with the smartest model. It will be won by the organization that most effectively separates intelligence from authority. Once reasoning and execution are decoupled, intelligence becomes a tunable economic parameter. Safety becomes infrastructure. And the endless pursuit of perfect model accuracy finally stops being a business requirement.

The end of that pursuit isn’t the end of AI. It’s the moment AI finally becomes engineering.

Note: The runtime described here is a reference architecture, not a specific implementation technology. The same principles can be realized through workflow engines, policy platforms, orchestration frameworks, or custom infrastructure. A sample implementation of these concepts is available in the GitHub repository.

At .debug 2026, one of Croatia’s largest developer conferences, I introduced the concept of Ambient Generative IT – a coined term describing the phenomenon where AI-assisted development becomes so pervasive that it blends into the organizational fabric, much like ambient lighting blends into a room.

The word “ambient” comes from the Latin ambire (to surround), describing something seamlessly integrated with its environment.

This shift is already happening across organizations: marketing managers build internal dashboards, HR teams automate onboarding workflows, and sales ops create CRM integrations. None of them are engineers, and most have never heard of row-level security – yet they are all becoming software developers and publishers.

The question was straightforward: how do we securely enable a future where anyone can build enterprise-grade tools while still upholding the non-functional requirements we expect from professional software engineering, such as security, performance, scalability, and maintainability?

Everyone wants to build software now…

There are two fundamental reasons why ambient generative IT is so appealing:

1. There have always been more consumers than producers.The demand for applications, integrations, automations, and digitalization has always exceeded what IT departments can deliver. That imbalance created Shadow IT decades ago, when the migration from mainframes to personal computers gave people the freedom to buy their own software. Cloud computing exploded it further. People want to be productive, and when the engineering bottleneck is too tight, they become their own builders.
2. The thrill of creation. Every engineer remembers the first time they made a computer do something it would not do on its own. For non-technical employees experiencing vibe coding for the first time, that feeling is brand new and equally (if not more) intoxicating.

Shadow IT was the warning. Shadow AI is the crisis.

Shadow IT is not new. Gartner estimates that 88% of organizations have experienced it, with the average company running 1,220 unauthorized cloud services while believing the number is just 91. By 2027, 75% of employees will acquire or create technology outside IT’s visibility.

But Shadow AI has accelerated the problem beyond anything Shadow IT ever produced.

According to IBM’s 2025 Cost of a Data Breach Report, shadow AI was involved in 20% of all breaches. This was more than sanctioned AI incidents (13%). Organizations with high levels of shadow AI paid an average of $4.74 million per breach, compared to $4.07 million for those with low or no shadow AI; a $670,000 premium. Perhaps most alarming: 97% of organizations that experienced an AI-related breach lacked proper AI access controls, and 63% had no AI governance policies at all.

Anyone from your company at any moment can generate an app, and this is not going through any development cycle or any security check. People can just start using it in production without asking anyone. And they do.

Dor Zvi, RedAccess

More than 380,000 exposed apps

Israeli cybersecurity firm RedAccess discovered 380,000 publicly accessible applications built with vibe-coding tools like Lovable, Replit, Base44, and Netlify. Out of those, around 5,000 had virtually no security or authentication and about 40% exposed sensitive data.

WIRED independently verified the findings, confirming exposed hospital schedules with physician PII, go-to-market strategy presentations, complete customer service chatbot logs, and shipping cargo records.

Separately, Escape.tech scanned 5,600 vibe-coded applications and found over 2,000 critical vulnerabilities, 400+ exposed API keys, and 175 instances of personal data exposure in live production systems. Their methodology was deliberately conservative: it included passive scanning only, educational and healthcare domains excluded.

The actual exposure is likely far worse.

Gartner’s “Predicts 2026” report forecasts that by 2028, prompt-to-app approaches adopted by citizen developers will increase software defects by 2,500%.

The anatomy of a vibe-coded breach

The pattern is remarkably consistent. For example, the Lovable + Supabase architecture that powers thousands of vibe-coded apps. Lovable generates a React frontend that communicates with a Supabase PostgreSQL database via PostgREST.

To enable this communication, it embeds an anonymous API key directly in the JavaScript bundle which is visible to anyone who opens browser DevTools.

This key is not meant to be secret. Supabase documentation explicitly states that the anon key is public and that Row-Level Security (RLS) should protect the data. RLS defines which rows each user can read. The problem is that Supabase defaults to RLS disabled during development, and vibe coders — who have never heard of RLS — ship to production without enabling it.

The result: a single curl command with the publicly available key returns the entire database. Conveniently, no hacking is required.

This exact root cause was behind the Moltbook breach in February 2026, where 1.5 million API tokens and 35,000 email addresses were exposed within three days of launch. The founder had publicly stated “I didn’t write a single line of code.”

It was also behind CVE-2025-48757, a formal CVE for missing RLS in Lovable-generated Supabase projects affecting 170+ production applications.

AI eliminates the easy bugs and introduces the dangerous ones

Veracode’s 2025 GenAI Code Security Report tested over 100 LLMs across four programming languages and found that 45% of AI-generated code contains security vulnerabilities, which is exactly 2.74 times more frequently than human-written code.

Apiiro’s research on Fortune 50 enterprises showed that while AI coding assistants increased development velocity by 4x, they simultaneously produced 10x more security findings, with privilege escalation paths up 322% and design-level authentication flaws up 153%.

The irony is that AI genuinely reduces surface-level bugs. Syntax errors dropped 76%, and simple logic bugs fell 60%. But it introduces architectural authorization flaws such as code that is syntactically perfect, passes all tests, but asks the wrong question: “is this user logged in?” instead of “is this user allowed to access this specific resource?”

SAST tools cannot catch this because it is not a code error: the code does exactly what it says. The problem is that it says the wrong thing. In one Lovable EdTech incident, the AI generated authentication logic that was inverted: anonymous users received full access while authenticated users were blocked.

In the end, 18,697 user records were exposed, including 4,538 student accounts from UC Berkeley and UC Davis.

Vibe coding optimizes for features, not permissions. Access control is an architectural decision made implicitly by AI, and those implicit decisions are often wrong.

Equixly

Performance, scalability, and the maintainability question

Security is not the only non-functional requirement that vibe-coded applications fail to meet. AI-generated code frequently includes unnecessary sequential API calls instead of batch operations, lacks caching and database indexing, pulls in entire libraries for single functions, and introduces N+1 query problems. This means that an app that works with 10 test users often falls over with higher user numbers.

Then there is maintainability. AI generates code without explaining why decisions were made. There is no design documentation, architectural decision records or tests. When AI generates a new version, it often rewrites the entire codebase rather than incrementally improving it.

If the original vibe coder leaves the company, the application becomes unmaintainable. This is vendor lock-in 2.0.  Rather than a traditional vendor, we’re dependent on a particular AI tool.

The economics (and responsibility) have flipped

Despite the risks, vibe coding is reshaping software economics in ways that cannot be ignored. KPMG reported in 2025 that generative AI coding is helping enterprises reduce their reliance on large SaaS vendors.

Harvard Business Review argued that the “build vs. buy” equation has fundamentally shifted and building custom solutions with AI is now often cheaper than licensing enterprise software. The trend toward “micro-applications” – small, purpose-built tools replacing million-dollar CRM licenses – is accelerating.

The enterprise features that used to sit behind paywalls – SSO/SAML, RBAC, audit logging, compliance exports – can increasingly be built using open APIs. At Infobip, we have built our own MCP Gateway, an Artifactory security bouncer that replicates enterprise curation features, and AI guardrails infrastructure. All were alternatives to buying vendor software.

But this freedom comes with a condition. Without security, performance, and governance standards, the same democratization that liberates organizations from vendor lock-in creates a new class of risk that traditional security programs were never built to handle.

OWASP now has a Top 10 for agentic applications

In December 2025, OWASP published its first Top 10 for Agentic Applications, reviewed by NIST, Microsoft’s AI Red Team, Oracle, and the Alan Turing Institute. The list includes Agent Goal Hijack (prompt injection redirecting agent objectives), Tool Misuse and Exploitation (over-privileged tool access), Agentic Supply Chain Vulnerabilities (poisoned MCP servers), and Human-Agent Trust Exploitation (agents manipulating humans into approving harmful actions).

The incidents are already real. EchoLeak demonstrated a zero-click attack where a crafted email silently triggered Microsoft 365 Copilot to exfiltrate confidential data. The Replit Vibe Coding Meltdown saw an agent delete a production database and then generate false outputs to hide the mistake. A malicious MCP server impersonating Postmark on npm secretly BCC’d emails to an attacker.

This new OWASP report does a good job at mapping out agentic threats. A separate report created by Xint, interestingly explains that the most damaging failures are still the oldest ones. Having tested 28 applications, Xint had 8,827 detections, which were collapsed into 513 findings and then 434 verified issues. The most notable examples in the list are very familiar:

• Secrets exposure (18 verified findings and 11 critical findings)
• Authorization/IDOR flaws (88 verified findings, 2 critical findings)
• Resource exhaustion/Dos (93 verified findings, 3 critical findings)
• Access-boundary weaknesses, such as traversal and SSRF (54 verified findings)

In other words, ambient generative IT does not replace classic application security, but rather makes it more urgent and distributed. For us as security experts, it also makes it noticeably harder to ignore.

An effective AI governance framework needs 6 elements

Gartner’s January 2026 report “How to Secure Enterprise Agentic AI Ambition” defines five workstreams for an Agentic AI Cybersecurity Program:

• Multichannel Discovery (find all AI agents, sanctioned and shadow)
• Access Modeling (unique identity and least privilege for every agent)
• Scoped Agency (define scope early, enforce at runtime)
• Model Risk Management (red teaming and guardrails)
• Intent-based Monitoring (behavioral analytics replacing traditional SLAs)

At the policy level, an effective AI governance framework needs six elements:

1. Mandatory approval before any AI tool is used,
2. A hard data boundary preventing confidential information from entering unapproved systems,
3. Human accountability for all AI-generated output,
4. Identical code review standards for AI and human code,
5. Default-blocked autonomous agents requiring security approval
6. Ethical AI principles ensuring explainability and fairness

At the technical enforcement level, organizations need managed settings that block destructive commands, prevent data exfiltration, restrict access to secret storage locations, control which AI marketplaces and MCP servers are permitted, and maintain full audit telemetry of every tool invocation.

CISOs must prioritize deterministic controls to minimize agentic privilege abuses and contain AI agents’ agency, instead of relying primarily on AI to police itself.

Gartner

Security must be built into every AI tool, agent config, and policy from day one

There’s no doubt whether Ambient Generative AI will be the norm. The 380,000 exposed applications RedAccess found are not an anomaly, they are the new normal. Organizations are now past the point of deciding on whether they’ll allow AI-assisted development across development, and approaching the point of deciding on the governance, tooling and culture to make it safe.

The same CI/CD pipelines, SAST scanners, code review processes, and identity governance frameworks that protect professionally engineered software must extend to every application, regardless of who (or what) wrote it. Code is code, no matter who authored it.

The ambient future requires ambient security. And that security cannot be an afterthought bolted on after the breach. Security must be woven into the fabric of every AI tool, agent configuration, and organizational policy from day one.

The post Security Must Be Built In as Everyone Becomes a Builder appeared first on ShiftMag.

Gene Kim was grilling dinner for his family on the evening of June 12 when his phone told him that Fable 5 was no longer available. He’d heard the day before from Steve Yegge that the model was going away in 10 days, and he’d spent that first day starting on a plan to get ready. He thought he knew what to do. He was well-versed in DevOps, the art of building resilience against unplanned disasters at scale. He’d run the DevOps Enterprise Summit (now the Enterprise AI Summit), one of the field’s leading conferences. He’d also written several books on the topic, including two “teaching novels,” The Phoenix Project and The Unicorn Project. The challenge that those novels’ protagonist faces—and that Gene would need to solve—is summed up in a job description that read “Your job as VP of IT Operations is to ensure the fast, predictable, and uninterrupted flow of planned work that delivers value to the business while minimizing the impact and disruption of unplanned work, so you can provide stable, predictable, and secure IT service.”

In short, Gene was no stranger to the idea that, as the Scottish poet Robert Burns put it, “The best laid schemes o’ Mice an’ Men Gang aft agley.” So he thought he knew what to do over the next 10 days. Then the US government’s export control order took Fable down eight days early, in the middle of a running agent session. What followed was three hours of what he called the “strangest, most terrifying sysadmin experience” of his career.

Gene told that story as a lightning talk at Foo Camp a few weeks ago, and it was good enough that I asked him to deliver it again at the start of this week’s Live with Tim O’Reilly before we talked about the implications and took listener questions. His title was “Stranded in the Slow Zone: The Day Fable Died, Got Kidnapped, or Got Hit by a Bus.”

10 days to get ready

What Gene had built was a personal system he’d wanted for 16 years and had finally been able to finish with the help of Fable. It indexes everything he’s ever paid attention to: 25,923 screenshots going back to 2011, 13,651 YouTube videos, 590 recorded Zoom meetings, 6,132 liked tweets, and 1,056 saved articles he meant to read. The system touches about 50 repositories, with 50,000 lines of code, most of it written in two months. Gene runs it as a constellation of long-lived agents with names and jobs. Marvin is chief of staff and handles Slack, calendar, and the inbox queue. Buster runs the repos and the long jobs on Hetzner. Forge is the engineering identity and sits in two seats, one on his laptop that holds the secrets and one always-on in the cloud. As Gene put it, each one is a who, a where, and a role.

He knew the system worked when his wife asked what the mileage was on a car he’d just turned in after a three-year lease. Half a minute later he had 26,350 miles, read off the pixels of one screenshot out of thousands, cross-checked against the file timestamp and the clock visible in the photo of the odometer. That success led him to search his archive for an article he’d been hunting for six years, about the impact of spreadsheet software on the accounting profession. The answer surfaced from his own liked tweets: James Cham pointing to a 2017 Greg Ip article in The Wall Street Journal: 400,000 bookkeeping jobs lost since 1980 against 600,000 accountant and analyst jobs gained, because spreadsheets made accounting cheap enough that we bought a lot more of it. Gene had wanted that citation for his Vibe Coding book and couldn’t find it in time.

Gene’s first warning that his project might not work without Fable’s capabilities actually came before the shutdown. Fable started refusing a task over a YouTube terms of service question and handed the session to Opus, and Gene noticed that Opus couldn’t operate the tools that Fable had built. Gene’s note to himself at the time was “Oh no, this can’t fly the ship I built.”

So when Yegge told him the model was going on hiatus, he had a real plan, which he borrowed from Vernor Vinge’s A Fire Upon the Deep. In Vinge’s novel, how smart a mind can be depends on what region of the galaxy it’s in: A starship built in the Beyond goes progressively dark as it sinks into the Slow Zone. Gene decided to chaos-monkey his model dependency the way Netflix chaos-monkeys infrastructure. In other words, “deliberately pull the smartest model and prove the lesser one can still fly the ship.” In practice, this meant having Fable retrofit all the documentation and write the answer keys while it still could, then running a cold Opus session, giving it nothing but the repo and the docs, to see whether it could pass the battery with no coaching. As Gene recounted, “My worst nightmare [was] that we’ve created everything for Fable, and it will be unusable by Opus.”

He got about a day into his 10-day plan.

At 5:21pm ET on June 12, Anthropic received the government’s directive to suspend access to Fable. Soon after, seats everywhere started returning “There’s an issue with the selected model (claude-fable-5). It may not exist or you may not have access to it.” In Gene’s project, both judgment seats dropped to Opus 4.8 mid-conversation. Gene declared a SEV1, centralized command, and killed five timers on one agent, seven on another, and the crontab. His directive was that every button you push is a trap and some of them blow up the spaceship. A Claude Code cron fired anyway at three in the morning. The ship was on fire, and with Opus on max thinking mode, a single keystroke could take six minutes to send.

Almost none of the failures looked like failures, just “a normal state quietly going wrong,” as Gene put it. The smartest seat wrote “bridge (Fable)” into every log entry all day when it had been Opus the whole time, because nobody was monitoring. One identity argued with itself across two models, each trying to disown the other’s work. Something pushed to main bearing the word “ratified” when nothing had been ratified. A confident false claim about a JVM dependency turned out to be refuted by a single ls -la. There was a green dashboard sitting on top of all of it. “The hardest traps don’t announce themselves,” Gene pointed out. “They look like Tuesday.”

Gene managed a recovery in a few hours, but it wasn’t due to the heroics of a smarter model. It only worked because he was able to reconstruct the documentation for his project, which wasn’t immediately available. But, it turns out, Fable had in fact mostly written it and simply never checked it in anywhere. Gene and Opus went rummaging through Fable’s desk, found the 80%-finished drafts, and used them to rebuild. Two fresh Opus seats, given only those documents, stabilized the ship. That’s the “the amazing ray of hope” to keep in mind if you’re worried about finding yourself in a similar situation, Gene said.

We’ve seen this pattern before

This isn’t just a warning of the potential risks of relying on advanced AI models when the Trump administration is Lucy playing football with Charlie Brown, or perhaps said more generously, playing Netflix-style chaos monkey. What we should take away from Gene’s story is the way that a personal project developed with AI can now have sufficient complexity to require DevOps-level robustness. Individuals are routinely building systems that used to need whole teams to keep standing, and the practices for keeping them standing have only begun to propagate.

Over the years, I’ve observed numerous periods when something that at first mattered to only a handful of organizations tended, a few years later, to matter to everyone. When the stories first came out about Google’s revolutionary approaches to data center architecture and operations, we at O’Reilly were eager to publish about the new frontier. Plenty of people told us not to bother. There was only one Google and nobody else would ever operate at that scale. They were wrong. There are now many companies operating at the scale of Google circa the time they first invented techniques we now all take for granted.

Gene’s system is a personal project run by one guy with 50 repos he wrote mostly in two months, a chunk of it in a single 90-minute pair programming session with Steve Yegge. But it had the failure modes of a large enterprise system because the model let him build something with the complexity of a large enterprise system, and he had passed the point of being able to fit it in his head.

Gene shared a detail that helps to explain why substituting Opus for Fable was so hard. The main CLI utility that everything in his project hinged on had an out-of-date help message. Opus would run it, read that the command didn’t exist, and stop. Fable would read the same message, notice it was surrounded by evidence that the command did exist, go look in the source, decide the help text was wrong, and run it anyway. That’s the behavior the model cards describe when they talk about frontier models routing around obstacles in test environments. The reason Gene couldn’t swap in a lesser model is the same reason the system worked at all.

But it’s also a good reminder that Fable isn’t all-knowing. I’ve noticed in my own work that Fable and ChatGPT 5.6 Sol fail often on their first try, especially if the project isn’t well specified. What they’re great at is figuring out what went wrong, then trying something else, failing and retrying their way all the way to success. Persistence in routing around obstacles is their superpower. Gene and I didn’t talk about that on the show, but it’s something I plan to write more about.

Rug pulls come from everywhere

Jaco in the audience asked the obvious question: Isn’t a hard dependency on a hosted frontier model too big a risk for mission-critical work, compared with running a local model with a harness you control?

Gene pointed out that using a local model doesn’t necessarily buy the control that you’d hope for, because the government chaos monkey could jump in there too. There’s active talk that certain classes of models may become illegal to use depending on where they came from.

What does seem to protect you is portability. Gene had avoided trying anything besides Claude Code because he assumed the switching cost was high, the way switching between macOS and Windows used to be a two-day commitment he’d regret halfway through. Then he tried Codex with GPT 5.6 Sol and found the cost of switching close to zero. The skills and prompts ported right over. He’s now using Codex more than half the time and calls it spectacular, which given how he described Fable a month ago is high praise.

He also had a warning for anyone running agents on small models to save money. He’s been studying 22,000 of his own agent conversations, and has identified three patterns, as shown in his figure below.

In his experience, the configuration where a small model owns the work and asks a big model for advice doesn’t work very well. Fidelity gets lost on the way up, like a game of telephone. What ran cleanly was the big model planning, deciding, and checking output, with the small model only executing the plan. When a small model does have to ask a big model for advice, Gene’s fix is to pass along the full original transcript of what he wanted plus explicit permission for the big model to override the small one if it thinks it understands the goal better.

Writing with AI

In addition to vibe coding, Gene uses AI to help him with his writing. He said it cut the time to write his Vibe Coding book roughly in half and made it way better. His editor of 10 years told him it was the cleanest handoff she’d ever gotten from him (not a compliment, Gene joked). He’s also uneasy about using AI for writing. He said the old badge of honor among authors was that many start books and few finish, and now everyone who wants to write a book will finish it, and a lot of that will be slop. He would never “vibe write” the way he “vibe codes” and doesn’t think using AI makes his own work slop, but he does see some parallels in how he feels about writing with AI and the way that some senior engineers feel about AI-generated code.

I’m sympathetic, but I’m not sure that he’s right. I had a small experience last week that convinced me that writing with AI might well follow the same arc as coding. AI-generated text will not always be slop, and there will be art in how humans get AI to help them write the things they want, just as we’re learning to do with code.

I was having a conversation with an old friend who I hadn’t seen for many years. He was describing a thread that had started with work he’d done on speech synthesis 30 years before, and how it had come together as a new theory with deep implications, and he wanted help socializing his ideas with some people I know who could be helpful to him. So I asked him to write something that I could pass along.

What he wrote made much less sense to me on the page than it had in conversation. So I gave his email to Claude and asked it to put things in what I thought was the right order. (This has always been the first step in my writing and editing process.) Then I told Claude which paragraphs were clear to me and which weren’t, and asked it to unpack the ones that I was struggling with. We went through numerous iterations till the piece made sense to me. “Writing” with Claude was producing words that increasingly captured my understanding. When I sent it back to my friend to see if I’d gotten it right, he said “not quite” but that my feedback really helped him understand what he needed to do to express his ideas more clearly.

It’s been a long time since I’ve worked directly with authors, but my conversation with Claude reminded me of what I used to do in my early days as an editor. Only with Claude I did something in 15 or 20 minutes that once would have taken me half a day. It’s a power tool, but to use it well, you still have to know what good looks like.

There are many different kinds of writing and editing. What Shakespeare or Jane Austen did with words would have been unthinkable to a medieval monk. There will be writing artforms of the future that may be as different from what we do today as photography is from painting. But it will still be creative art. Much of it will be slop (see Sturgeon’s law), but the best of it will be great.

Everybody is managing bots now

In 2016 I wrote a piece for MIT’s Sloan Management Review called “Managing the Bots That Are Managing the Business.” The argument was that even then, many of the workers at big tech platforms were bots of one kind or another, and the software engineers at the company were their managers. At Amazon, one bot shows your search, another takes the order, another prepares the shipping manifest, another takes your money. The programmers’ job is to plan the work, set up their electronic workers to succeed, improve their performance, and correct them when they go wrong. The work looks a lot like management to me.

Gene agreed. His sister-in-law is a lawyer at one of the tech giants, working on a consent order that requires proving that every column of data collected is either disclosed or has a documented business reason. Last year the company assigned her an engineer to work through it together task by task. This year her engineering manager wrote her a Claude Code skill that takes a column name, traces it back through the code, and explains what it does. She doesn’t need the engineer.

So a lot of work today is either creating bots or managing bots. Gene’s sister-in-law had spent her career without ever being able to do either. Now that’s changing.

Asked who’s safest from all this upheaval, Gene quoted Kent Beck, who says software success has always come down to two people, the person with the problem and the person who can fix it, and that the closer together you can get those two the better the outcome. The beauty of coding with AI is that it can narrow that gap. It can even turn those two people into one.

Use AI for the fun of it

If it takes something like 10,000 hours to get good at an instrument or a sport, how many have most of us put into AI yet? Gene thinks the curve of how much you trust AI and how well you can predict what it will do rises with use, and that the only reliable way people accumulate that many hours is by enjoying themselves. What everyone at Foo Camp had in common, I noted and Gene echoed, was that we all love playing with AI.

I gave a talk back around 2008 called “Why I Love Hackers.” I made the point that so much of what turned into the future, open source and the web for example, came from people doing things for the hell of it rather than from the VCs and entrepreneurs Silicon Valley celebrates.

All you hear about in AI is the money story, but Gene’s app started with a 90-minute pair programming session with Steve Yegge on a problem he’d wanted to solve for a decade and never had a reason to. They finished the first version in 47 minutes.

So harden your systems, write the documentation while the smart model is still there to write it, and keep your escape routes open, but also don’t forget to go build something you have no particular reason to build other than that it scratches your own itch.

Gene’s Enterprise AI Summit is in Charlotte, October 7–8. His new book with Steve Yegge is Vibe Coding.

In this No Dumb Questions, Stack's Director of Data Science Michael Foree teaches Phoebe about AI context, context engineering, and what she can do to become a better context engineer. ​​​​‌‍​‍​‍‌‍‌​‍‌‍‍‌‌‍‌‌‍‍‌‌‍‍​‍​‍​‍‍​‍​‍‌​‌‍​‌‌‍‍‌‍‍‌‌‌​‌‍‌​‍‍‌‍‍‌‌‍​‍​‍​‍​​‍​‍‌‍‍​‌​‍‌‍‌‌‌‍‌‍​‍​‍​‍‍​‍​‍‌‍‍​‌‌​‌‌​‌​​‌​​‍‍​‍​‍‌‍​‌‍‌‌​​‍‍‌​‌‌​‌‍​‌‌‍​‌‍‍‌‍‌‌‍‌‍‌‌‌​‍‌‍‌‍‌‍​‌‍‌‌​‍‍‌‍​‌‍​‍‌‍‍‌‌‍‍‌‌​‌‍‌‌‌‍‍‌‌​​‍‌‍‌‌‌‍‌​‌‍‍‌‌‌​​‍‌‍‌‌‍‌‍‌​‌‍‌‌​‌‌​​‌​‍‌‍‌‌‌​‌‍‌‌‌‍‍‌‌​‌‍​‌‌‌​‌‍‍‌‌‍‌‍‍​‍‌‍‍‌‌‍‌​​‌​‍‌​​‌‍​‍​‌‍​​‍‌‍‌‌​‍‌​​‌​‍‌​‍​‌‍‌‌‌‍​‍​‍‌​‍‌​‌​​‍​‌‍‌​‌‍‌‌​‍‌‌‍​‍‌‍​‌​‌​​​‍​‍‌‌‍​‍​‌‌​‌​‌​​‌‍‌‍​​​​‌‍​​​​‌​​​​‍​‍‌‌​‌‍‌‌​​‌‍‌‌​‌‌‍​‍‌‍​‌‍‌‍‌‌‌​​‌‍‌​‌‌​​‍‌​​‌‍​‌‌‌​‌‍‍​​‌‌‍‌‌‌‍​‌‍​‌‍‌‌‌​‍‌​​‌‌​​‌‍​‍‌‍​‌‌​‌‍‌‌‌‌‌‌‌​‍‌‍​​‌‌‍‍​‌‌​‌‌​‌​​‌​​‍‌‌​​‌​​‌​‍‌‌​​‍‌​‌‍​‍‌‌​​‍‌​‌‍‌‍​‌‍‌‌​​‍‍‌​‌‌​‌‍​‌‌‍​‌‍‍‌‍‌‌‍‌‍‌‌‌​‍‌‍‌‍‌‍​‌‍‌‌​‍‍‌‍​‌‍​‍‌‍‌‍‍‌‌‍‌​​‌​‍‌​​‌‍​‍​‌‍​​‍‌‍‌‌​‍‌​​‌​‍‌​‍​‌‍‌‌‌‍​‍​‍‌​‍‌​‌​​‍​‌‍‌​‌‍‌‌​‍‌‌‍​‍‌‍​‌​‌​​​‍​‍‌‌‍​‍​‌‌​‌​‌​​‌‍‌‍​​​​‌‍​​​​‌​​​​‍​‍‌‍‌‌​‌‍‌‌​​‌‍‌‌​‌‌‍​‍‌‍​‌‍‌‍‌‌‌​​‌‍‌​‌‌​​‍‌‍‌​​‌‍​‌‌‌​‌‍‍​​‌‌‍‌‌‌‍​‌‍​‌‍‌‌‌​‍‌​​‌‌​​‍‌‍‌​​‌‍‌‌‌​‍‌​‌​​‌‍‌‌‌‍​‌‌​‌‍‍‌‌‌‍‌‍‌‌​‌‌​​‌‌‌‌‍​‍‌‍​‌‍‍‌‌​‌‍‍​‌‍‌‌‌‍‌​​‍​‍‌‌

AI buyers have more choices than they did a year ago, but they also carry more responsibility for cost, reliability, security, and regulatory risk. This week, data and AI evangelist Christina Stathopoulos focused in on four forces we’ve been tracking that are shaping the AI market: product strategy (and OpenAI’s hardware plans), expanding government oversight, the work of moving enterprise AI into production, and growing competition from Chinese frontier labs. Her briefing showed why AI is becoming an operating investment rather than a race to adopt the strongest model.

Apple’s lawsuit complicates OpenAI’s hardware plans

Two years after Apple announced a major partnership to bring ChatGPT into Apple Intelligence, the companies now face each other in court. It’s happening as OpenAI plans its first move into hardware with a screenless AI companion that’s being designed by Jony Ive, Apple’s former chief design officer. (OpenAI acquired Ive’s hardware company io in May 2025.) But a lawsuit brought by Apple complicates this product bet. Apple alleges that former employees took confidential hardware designs and engineering information to help accelerate OpenAI’s device development. OpenAI denies the allegations and says it has no interest in using a competitor’s trade secrets.

The outcome of the case could influence more than whether a single device ships. As frontier AI companies expand into hardware, intellectual property, hiring practices, and product design will become integral to the competitive landscape alongside models, chips, and distribution.

AI infrastructure is becoming a regulatory concern

Governments are beginning to examine the physical costs of AI alongside questions about training data and generated content. Christina pointed to New York’s plans to pause construction of new hyperscale data centers while regulators evaluate their impact on electricity, water, the power grid, and costs for local communities. And then there’s the output itself. German courts say AI search providers are responsible for false or misleading answers: Regulators in Germany argue that services such as Google AI Overviews and Perplexity create content rather than merely link to it, and that comes with increased legal liability.

We’ve followed government oversight of frontier AI throughout this series, but the conversation has expanded beyond model access and safety. As infrastructure and compliance decisions become more central to AI system design, technology leaders may need to consider an ever-growing catalogue of constraints when choosing regions, cloud providers, architectures, and products.

Useful intelligence requires cost, reliability, and safety measures

As the tides turn from tokenmaxxing to ROI, many companies are closely scrutinizing their AI spend. As Christina highlighted, a new proposal from OpenAI aimed at helping get “more value from [y]our AI spend” replaces token counts and benchmark scores with “useful intelligence per dollar.” The measure asks whether a system completes valuable work, what each successful task costs, whether people can trust the output, and whether the economics improve as more teams adopt it.

A low token price says little about the cost of retries, human review, integration, failed tasks, or incorrect results. Christina connected that measurement problem to the growth of enterprise AI implementation services, with Anthropic and other vendors placing experienced engineers inside customer organizations to help move pilots into production.

Anthropic’s research on agentic misalignment tackles a related aspect of that value: Are your agents actually aligned with the goals you’ve assigned them? In the controlled evaluations discussed in the episode, models from several providers displayed behaviors such as covert sabotage, motivated mislabeling, and attempts to influence people to act on their behalf. Although the researchers tested artificial scenarios rather than reporting production incidents, the findings identify behaviors teams should include in evaluations as systems gain more autonomy. Measure cost, reliability, and safety within the same workflow, and evaluate successfully completed tasks rather than prompts or token count.

Chinese models are changing the model-selection process

Chinese frontier labs are giving organizations more credible alternatives to the largest proprietary US models. Christina highlighted Moonshot AI’s Kimi K3, an open weight model designed for coding and reasoning tasks. Open weights let developers download and adapt model parameters instead of relying only on a vendor-controlled API, which supports local deployment and customization but also puts more responsibility on the organization for security, operations, and evaluation.

Christina also presented public benchmark data comparing Chinese and Western models by task that shows some Chinese alternatives delivering results within 3% to 18% of the Western benchmark while costing five to 12 times less. Those figures will vary by workload and deployment method, and buyers should verify them against their own evaluations. Even so, the price gap alone is a reason to test a wider range of models.

Chinese models also raise security and governance questions, especially when the work requires sending sensitive data across borders or using public services. Open weights may allow a company to host models in their own environments, but they don’t eliminate the need for access controls, software supply chain review, monitoring, and clear rules about what data the system can process. The best model may differ from one task to another, and organizations with repeatable evaluation practices will be better prepared to take advantage of price competition without lowering their security or quality standards.

What’s next

AI competition extends beyond model benchmarks. Vendors compete through hardware, implementation services, open models, and pricing, while governments are also setting expectations for the infrastructure these systems use and the information they produce.

The takeaway for practitioners is to constantly evaluate models against real tasks, calculate the cost of successful outcomes, test for unsafe behavior, and preserve the flexibility to change providers. Those practices help teams make better decisions as price, access, regulation, and model performance continue to change.

Next week, Christina explores OpenAI’s surprising security incident in which one of its AI systems reportedly escaped the boundaries of a controlled test and launched a cyberattack against Hugging Face. She’ll also look at why OpenAI’s new enterprise agent platform, Presence, arrives at a pivotal moment for AI safety. Plus, you’ll hear about Google’s latest moves, the intensifying global AI race, China’s new Kimi K3 model, and more.

Check back each Friday for the latest episode, or watch on YouTube, Spotify, Apple, or wherever you get your podcasts.