https://clearmeasure.com/developers/forums/

Michael Nygard advises consulting firms, private equity teams, CTOs, CEOs, and boards when they require senior technology judgment for limited-term, high-impact situations — including architecture assessment, platform rescue, cloud and data cost intervention, AI engineering enablement, technical diligence, divestiture and carve-out architecture, and operating-model redesign. Over a 35-year career, he has worked at the seam where people, processes, organizations, and the systems they build intersect. Most organizations treat those as separate problems. The hardest failures, and the most consequential wins, live precisely where they interact. That through-line is what Release It! is fundamentally about. The vocabulary it introduced — circuit breakers, bulkheads, stability patterns — is now standard in how the industry discusses reliability, and the book is widely cited as foundational to DevOps and cloud-native practice.

At Nubank, he led the Data Business Unit with over $300 million in annual spend, then served as Chief Architect with reach across 2,500 engineers while the customer base grew from 75 million to 125 million across Brazil, Mexico, and Colombia. Results included cutting data-platform spend roughly 50% year-over-year, improving on-time data availability past 99%, building governance aligned with LGPD, GDPR, and CCPA, moving team engagement from the bottom decile to the 60th percentile, and rolling out AI coding tools to more than 90% of engineers without customer-visible quality regression.

At Sabre, as part of the CTO office, he helped lead development-practice modernization, GCP migration strategy, mainframe offload architecture, technical diligence, and divestiture architecture across thousands of applications and hundreds of products. He is most effective when the stakes are real, the system is sociotechnical, and the solution must hold across architecture, execution, economics, and organizational behavior.

LinkedIn: https://www.linkedin.com/in/mtnygard/
Personal Blog & Website: https://www.michaelnygard.com
GitHub: https://github.com/mtnygard Twitter/X: https://x.com/mtnygard
Release It! (Pragmatic Programmers): https://pragprog.com/titles/mnee2/release-it-second-edition/
97 Things Every Software Architect Should Know (O'Reilly): https://www.oreilly.com/library/view/97-things-every/9780596800611/
Goodreads Author Page: https://www.goodreads.com/author/show/6089.Michael_T_Nygard
LinkedIn Articles: https://www.linkedin.com/today/author/mtnygard Presentations Archive: https://github.com/mtnygard/presentations/wiki

Want to Learn More?
Visit AzureDevOps.Show for show notes and additional episodes.

On episode 524 James and Frank dive into the new .NET MAUI developer stack—covering the MAUI CLI/Maui Doctor that auto-provisions SDKs and emulators, the Maui Sherpa GUI for device/Xcode/provisioning management, and DevFlow's MCP server that lets AI agents inspect, interact with and automatically test apps (closing the loop). They also highlight the VS Code MAUI agent/skills, profiling tools, and the shift to core CLR in .NET 11 with performance tradeoffs to watch.

Follow Us

• Frank: Twitter, Blog, GitHub
• James: Twitter, Blog, GitHub
• Merge Conflict: Twitter, Facebook, Website, Chat on Discord
• Music : Amethyst Seer - Citrine by Adventureface

⭐⭐ Review Us ⭐⭐

Machine transcription available on http://mergeconflict.fm

Support Merge Conflict

Links:

• DevFlow
• MAUI Sherpa

Building a WordPress plugin was never part of Grant Fritchey’s comfort zone — but that’s the whole point of this series. In Part 5 of his AI-driven app build, Grant uses Microsoft CoPilot and GitHub Copilot to generate a working WordPress plugin for dbRosetta almost entirely through prompts. By the end, there’s a live, searchable web page – but also a growing list of lessons about what AI can – and can’t – do on its own.

Once again, this part of the series takes me into unfamiliar territory. I’ve never built a PHP plugin or a WordPress page…but that’s exactly what AI is for. So, let’s get into it.

This is the fifth part of Grant Fritchey’s series on building an app and database almost entirely through an LLM. The repo, including every prompt, lives on GitHub.

Creating the WordPress plugin with AI: the prompts I gave (and what I got back)

For anyone new here: I write prompts to Microsoft CoPilot (the paid version, running on my desktop), which then generates a prompt for Visual Studio (VS) Code and the GitHub Copilot agent.

There’s supposed to be a strict prompt structure we agreed on at the start of the series, but MS CoPilot keeps forgetting it. I’ve found that the more you try to correct AI on a given path, the more it drifts towards hallucination — so I’ve stopped fighting it, and just try to be more explicit upfront.

Here’s the exchange for this step:

The articles are all my own writing, but I do show some AI responses throughout. Those will always be clearly marked, so you’ll know which is me, Grant (nominally human), and which is the large language model (LLM).

Creating the WordPress plugin with AI: the actual creation (did it work?)

With the prompt pasted in, VS Code didn’t ask a single question — it just generated a complete WordPress plugin. Per my instructions, it also listed the next steps I needed to take manually.

I know how to do these steps but, since I want to lean on AI as much as possible, I went back to MS CoPilot and asked it how to copy the plugin folder across to WordPress. It gave me detailed instructions about file permissions, FTP/SFTP, and command-line copying.

Then I looked at WordPress and found a simple “Install Plugin” button. All I needed was a .zip file. I went back to MS CoPilot and it confirmed as much:

“Yes — that’s actually the most straightforward way to install a custom plugin you’ve built in VS Code.”

Which then begs the question: if it’s the most straightforward way, why didn’t it lead with that in the first place?

This is something I keep running into. It’s not just about giving AI the right context and prompt. After all, MS CoPilot did know we’re working in WordPress, and knew we had a plugin ready. But when I asked “how do I copy the folder?”, it answered exactly that question — and nothing more. It couldn’t step back and say “actually, here’s what you should really do.” It just goes: “OK. Here we go. WHOOP.”

Regardless, I now have a plugin installed and activated. I liked the credit line it gave itself: “By dbRosetta Team.” Ha — me (Grant) and two AIs!

Now, time to get the JWT setting, whatever that is…

And then all hell breaks loose.

Fixing the GitHub Actions deployment

Unbeknownst to the AI (but not to me – I already knew), the GitHub Actions deployment had been failing. When we went to get the JWT (JSON web token), VS Code discovered the deployments weren’t running — and also that they were only triggered on the Part 5 branch. We had to go back to that branch to start fixing things.

This kicked off a long back-and-forth session. I was copying and pasting error messages. The AI was updating and modifying scripts. First, the linting was so strict it was failing chunks of the generated code and, rather than fix the code, the AI loosened the lint rules – a classic development move!

Then, the other undertested code surfaced more issues. It was a long one. But after many iterations and several code updates (the AI doesn’t always remember prior context when generating new code), we successfully deployed the API to Azure.

Now, let’s get to testing the plugin. First: the dbRosetta web page.

The comical simplicity of using AI to set up my web page

Setting up the web page was almost comically simple. I created a new page, added [dbrosetta-search] to it, tested the URL, and it loaded. Done! I have a functional web page built almost entirely by AI.

Unfortunately, though, I hate how it looks. This was my first real view of what the user interface (UI) prompt produced. We’ll iterate. But first, let’s test the search…

<BOOM> Error. Here’s how the AI handled it:

“OK. The page loaded and the dbRosetta plugin was visible. When I typed in a search term, I got this error: body/filters must be object”  The WordPress plugin is using the wrong API request format. Let me check the API client code and fix it… I see the issue. The plugin is sending filters as an array, but the API expects an object (key-value pairs)… The field is canonicalTerm, and filters should be an object. Let me fix the WordPress plugin.”

Once prompted with the error, it diagnosed the problem, read the relevant files, and fixed the code. It just couldn’t remember what it had built without being nudged. I updated the plugin and, this time, it worked.

Conclusion: it was a success (albeit a painful one)

I’m calling this a success. Yes, there was pain. Yes, you can’t just let the AI run without input, hand-holding, guidance, and the occasional smug “I told you so” feeling. But it did the job — and did it pretty well.

There’s work ahead. I need to refactor the UI to make it actually correct — right now it should let you specify where a term is from, and return results either for all databases or a selected one. I also want the AI to load up the test data.

Ahead of the next steps, I have to answer the question: should I let the AI decide what to build next to make this a fully functional application? Or do I wait to see if anyone wants to start issuing pull requests?

In summary: what I learned from building a web page with AI

AI answers the questions you ask – not the ones you should have asked.
When I asked how to copy a plugin folder, MS CoPilot gave me a detailed FTP walkthrough. The simpler answer — just zip it and use WordPress’s built-in upload — was never mentioned until I stumbled across it myself. It can’t reason around your question to a better solution.

Context doesn’t persist the way you’d hope.
The AI forgot what it had built, introduced a format mismatch between the plugin and the API, and only diagnosed it correctly once I fed it the error message. It then fixed it quickly — but it needed the prompt.

The deployment pipeline will bite you when you least expect it.
A failing GitHub Actions workflow quietly sat there until mid-build. Having a test scaffold in place made fixing it far less painful than it could have been.

Getting something working is fast – but getting it right takes longer.
A functional web page in minutes is genuinely impressive. But “functional” and “correct” aren’t the same thing — the UI still needs a refactor, and the data isn’t loaded yet.

The junior developer analogy keeps proving itself.
AI is fast, capable, occasionally forgetful, and very literal. Bottom line: you still need someone experienced in the room.

The last word: CoPilot’s assesment of where dbRosetta is up to at the end of part 5

At this stage, dbRosetta has moved beyond its foundation of schema translation and API design into the practical challenge of presenting those capabilities through a WordPress plugin and web interface. While this step has pushed the project into less familiar territory, it marks an important milestone: the toolkit is no longer just a backend service, but something contributors and users can interact with directly.

The work so far has established a reproducible path from database migration scripts to a functioning plugin, and although there are still refinements ahead, the project now stands as a tangible, teachable reference implementation that bridges CI/CD workflows with accessible front-end integration.

The post AI built my WordPress plugin. Here’s what worked – and what went wrong (building an app with AI, part five) appeared first on Simple Talk.

Thomas Betts talks with Clare Liguori, the technical lead on the open source Strands Agents SDK. The conversation covers how Strands Agents has grown from a Python SDK to a full agent harness running in production. Clare shares some lessons learned from building agents at scale, shifting to a model-driven architecture, and what comes next as the LLMs that underpin agents continue to improve.

I was at a ThoughtWorks “The Future of Software Engineering” event in Switzerland some weeks ago. What a braintrust. Lots of conversations about AI generally, agentic AI in particular and harness engineering.

Here’s a distinction I’ve been mulling for a while and keep reaching for in conversations about AI systems in the enterprise: build-time AI versus run-time AI. One is artifact-oriented, the other is execution-oriented.

Run-time AI isn’t only transactional, either. Generation of movies, songs, images, and slide-shows is run-time too. Those outputs are files, which makes them look artifact-like, but they’re products of execution, not engineering artifacts. Say a generated slide-show doesn’t specify the behavior of any system, and nobody’s putting it through code review as it might be going straight into GoogleDocs or Office 365. That’s the real test for which side of the line something sits on: build-time artifacts specify (later) behavior; run-time outputs are the consequences of wished-for behaviors.

Cross that with the conversational -> workflow -> agentic ladder that has become the common way of classifying AI systems, and you get a tidy 3×2:

Harness engineering is a build-time discipline

The interesting consequence: harness engineering is predominantly a build-time discipline, even though it exists entirely to influence run-time behavior.

The build-time harness looks like source code, because it is source code:

AGENTS.md
tools.yaml
memory_policy.py
planner.py
evals/
guardrails/
prompts/

Well, it is today. It gets committed to Git, reviewed in pull requests, and deployed via CI/CD - the full software delivery treatment.

Run-time is a different world entirely. Steps in a workflow:

1. Customer asks for a refund
2. Agent plans
3. Looks up CRM
4. Issues refund
5. (Audit log written)
6. (Database updated)

The last two are in parentheses because the agent isn’t doing them directly - the existing application or service (AI made or not) it invoked does those as a matter of course.

No Git commit occurs during that interaction. The agent changed the state of several systems of record, but it didn’t change itself.

An orthogonal axis, not a maturity stage

It’s tempting to present build-time vs run-time as another rung on the maturity ladder, after conversational, workflow, and agentic. I’d resist that. It’s an orthogonal axis. There are two independent questions to ask of any AI system:

1. What kind of AI system is this? - Conversational → workflow → agentic.
2. Where does the “engineering” happen? - Build-time artifacts ↔ run-time execution.

Every cell in the 3×2 above is a legitimate place to be, with the non-agentic ones included in that claim. A prompt-engineered conversational system with its templates under version control is not “less mature” than an agentic one. Indeed, it may be exactly the right amount of machinery for the job.

What a “harness” actually is

This framing also gives “harness” a satisfyingly software-engineering flavor. Edward Mangini’s recent Harness Engineering: The Devil is in Your Details defines a harness very broadly - essentially “everything except the model.”. He credits others (Trivedy & Birgitta Böckeler) for that. I think there’s a cleaner separation: the harness is the version-controlled, deployable specification of agent behavior. The live conversation, the database state, and the external side effects are not the harness - they are the consequences of the harness executing.

That gives harnesses a lifecycle human software engineers will recognize:

**author -> review -> commit -> deploy -> execute -> observe -> revise

Everything before “execute” is build-time. “Observe” is where run-time telemetry feeds back into the next build-time iteration. Which is to say: harness engineering is just engineering, and all the disciplines we already have for source code - code review, trunk-based development, continuous delivery, configuration as code - apply to it directly.

Well of course, build-time vs run-time isn’t common language in the AI field, so I’ll keep on trying to update my mental model.

An AI agent pursues a goal on its author's behalf over minutes, hours, or days. It calls tools, writes files, keeps state between steps, and makes its own decisions within the guardrails it was given. At different moments it leverages a request/response service, a batch job, a serving endpoint, and other capabilities.

Running this type of workload safely requires a solution architecture of its own. In this article, we describe that solution as a blueprint to use for your agentic workload delivery needs.

A new workload class

The model is a stateless inference service. The harness turns it into an agent: it owns the loop, the memory, the context window, and the tool calls. The sandbox runtime decides what the agent may touch: which files, which binaries, which network endpoints. These three components have separate lifecycles, scale differently, and carry different security requirements, so we keep them separate.

Red Hat AI, the enterprise platform for any model and any agent across the hybrid cloud, is a natural host for that separation. We write this blueprint mainly from an Agent-as-a-Workload perspective. This means each agent is one sandboxed pod and one individual, independent value proposition. A second deployment pattern, Agent-as-a-Service, moves the loop out of the agent software into a shared runtime. Its adoption will follow business and operational needs, and we contrast the two later in this article.

We describe the solution blueprint as an open architecture of interchangeable components. Every component in Figure 1 is a role that several implementations can fill. AI accelerator vendors ship implementations for many of them; NVIDIA currently has the most visible set with NemoClaw, OpenShell, Nemotron, and CUDA-X. They appear in our blueprint on those merits, though they are not required. The architecture stands on open interfaces: an OpenAI-compatible inference access API and the open source Open Responses specification for models, Model Context Protocol (MCP) for tools, the agent-sandbox API for isolation, and SPIFFE (Secure Production Identity Framework For Everyone) for workload identity. This architecture gives each agent a verifiable, short-lived credential instead of a static API key.

We wrote this article for platform engineers and AI infrastructure teams who need to run autonomous agents safely on open, scalable AI platforms such as Red Hat AI. This article is not a tutorial for building an agent. Instead, it serves as a blueprint for operating agents as governed cloud-native workloads.

The solution architecture

In Figure 1, we lay out the complete blueprint in a single view. The rest of the article walks you through the blueprint component by component. The figure reads as three horizontal bands with one checkpoint between them, and the legend colors mark the planes.

The control band across the top (blue) creates and governs agents. Read it from left to right to understand the lifecycle:

• An agent blueprint declares what each agent is made of.
• The sandbox control plane turns that blueprint into a Sandbox resource.
• A controller turns the resource into exactly one pod.
• The workload identity service issues every pod and gateway a verifiable credential.

The dashed arrows dropping out of this band are control traffic, and nothing else about the figure moves until they've done their work.

The middle band is where the magic happens. It contains three zones from left to right:

• Skill backends: These backends can run anywhere (represented by a dashed border because location is not a commitment).
• Agent pod: This pod hosts the agent, harness, supervisor, and thin skill clients.
• Inference plane: This plane serves models inside the cluster and contains the gateway, semantic routing, scheduler, and vLLM workers.

Two gray boxes on the right edge sit apart from that flow on purpose: an optional external large language model (LLM) that is reachable only with masked personally identifiable information (PII), and audit and tracing, where every hop in the architecture eventually lands.

The checkpoint between the bands is the MCP Gateway. It's drawn in control-plane blue because of its function: every tool call leaves the pod through this gateway, is authorized by token claims rather than by the model itself, and then reaches a skill backend.

The platform bars at the bottom carry everything above them: Red Hat OpenShift, the Linux kernel, and the silicon. The silicon bar lists several options deliberately because the architecture assumes no single vendor.

Components

Each solution component in and around Figure 1 has a role, a duty, and a set of implementation options. We list these options in Table 1. Maturity varies across these options, and we separate production-grade implementations from technical preview features later in this article (see Table 4).

Agent packaging

The sandbox layer on Kubernetes consists of two pieces, with one on each side of the pod. Outside the pod sits a sandbox control plane, and inside the pod runs a supervisor process. In our blueprint OpenShell supplies both, and any implementation can fill either role because the contract between them is a Kubernetes custom resource.

The delivery path has two hops, with a custom resource in the middle. The control plane receives a request for an agent and writes a Sandbox custom resource; in our blueprint the OpenShell Gateway is the target implementation of this role as its Kubernetes path matures. The Agent Sandbox controller, a Kubernetes Special Interest Group (SIG) Apps subproject that implementations target, reconciles that resource into exactly one pod. One sandbox maps to one pod in the current API.

Inside the pod, the supervisor runs as PID 1 and starts before the agent does. It fetches policy from the control plane, installs the kernel filters, injects credentials, and then launches the harness as its child process. All egress leaves through the supervisor's proxy.

Policy enforcement splits along this same boundary. Kubernetes governs the pod from the outside using the following primitives:

• Pod security standards
• SELinux
• NetworkPolicy
• An optional Kata microVM through RuntimeClass
• Confidential containers beneath those layers

The supervisor governs the process from inside, regulating which binaries the process can run, which paths it can touch, and which endpoints it can reach. Stacked together, these layers provide three nested isolation rings: hardware or virtual machine (VM), pod, and process. Every primitive in all three rings relies on standard Linux and Kubernetes technologies, such as seccomp, Landlock, namespaces, SELinux, AppArmor, or Kata Containers. None of these primitives depend on a GPU vendor.

One discipline from our 12-Factor Agent article carries over directly: the agent workload is long-lived, but its sessions are not. Sessions remain ephemeral, while workflow state lives outside the pod in a durable data store, such as a database, an object store, or a durable-execution engine. Consequently, each new session starts with a clean context.

Agent-as-a-Service

Everything above assumes the loop ships inside the pod with the harness. Red Hat OpenShift AI 3.5 (Early Access at the time of writing) ships a second pattern through the open source OGX project (formerly Llama Stack): Agent-as-a-Service. Here the agentic loop runs in a shared runtime behind the Responses API. The client sends one request naming the goal and the MCP servers it requires. The loop runtime advertises the tool schemas to the model, executes the calls the model selects, feeds results back, and iterates to a final answer. Every step lands in a durable data store. Figure 2 contrasts the two patterns side by side.

The shared runtime inverts the pod pattern's trade-offs. It gains:

• Enforcement in one place: every action transits the loop runtime.
• One audit trail across all tenants.
• Credential isolation: clients never hold tool credentials.

It pays with:

• Shared blast radius: the loop service is a single point of failure.
• API mismatch: a request-scoped API fits long-running, goal-holding agents poorly.
• Blurred identity: per-agent identity survives only if the runtime propagates caller identity downstream, a gap SPIFFE does not yet address; the active standards work is in the IETF Workload Identity in Multi-System Environments (WIMSE) working group.

Most harnesses today are thick clients that own their loop, so adoption starts with operations teams standing up a shared runtime for request-scoped workloads: assistants, copilots, high-tenancy tools. Autonomous agents follow later.

The two patterns also compose. A sandboxed agent pod can delegate tool execution to a shared loop while keeping its own identity and rings, and the MCP Gateway's claims-based checks preserve per-agent policy inside the shared runtime. Duration and ownership pick the pattern, the same way you choose between a standard Deployment and a Knative service.

The governance

Governance in this blueprint consists of two mechanisms working as one system: workload identity establishes who the agent is, and tool authorization decides what that identity can do. Neither mechanism consults the model, and that's the point. While the agent's behavior can be manipulated, its credentials and permissions cannot.

Workload identity

SPIFFE/SPIRE gives each agent an automatically rotated cryptographic identity that is scoped to its namespace and service account. An authentication sidecar uses that identity for standards-based exchange with an identity provider such as Keycloak, using token exchange (RFC 8693) or the JWT bearer grant (RFC 7523), depending on the provider's support for externally issued tokens. 

This process trades the identity for short-lived access tokens to downstream services, to help verify no static API key exists anywhere in the path. The payoff is clear: a leaked prompt or workspace file contains nothing durable to steal. Lifecycle tooling planned as part of Red Hat AI injects the sidecar and the identity plumbing at deploy time, with no agent code changes.

Tool governance

The MCP Gateway, built on Envoy with Kuadrant/Authorino for policy, fronts every MCP tool server behind a single endpoint. The agent is configured with one MCP_URL variable and sees an aggregated tool catalog. Token claims minted during the workload identity exchange then decide which tools the agent can actually call. The gateway never reads the prompt. A prompt-injection attack that forces the model to call an unauthorized tool fails at the infrastructure layer because authorization is not the model's decision to make. 

What the gateway cannot stop is misuse of tools the agent is entitled to call, such as an injected instruction that exfiltrates data through a permitted endpoint. That risk belongs to the inference guardrails and egress policy described in the security model, which is why this checkpoint is one control among several rather than the whole defense. The same claims-based check covers every tool backend equally: a CUDA service, an OR-Tools container, or a Software-as-a-Service (SaaS) endpoint. MCP itself now lives under neutral governance, donated to the Agentic AI Foundation under the Linux Foundation in late 2025, with an authorization model that makes each tool server an OAuth 2.1 resource server, which is precisely the claims-checking this gateway enforces. This is the checkpoint from Figure 1 doing its job.

Inference routing

An inference request crosses three routers on its way to an AI accelerator. Three tiers exist because the three decisions have different owners: where a query may go is a security decision, which model serves it is a cost decision, and which replica executes it is an efficiency decision. Collapsing them into one router would put one component in charge of three concerns that change at different speeds and answer to different teams. Table 2 summarizes the three tiers, and Figure 3 follows one request across them.

Tier 1 is the security tier and lives client-side in every sandbox, inside the supervisor. Confidential context never leaves the cluster, and anything routed to an external endpoint is disguised to protect personally identifiable information (PII) first. For in-cluster traffic, the same router directs requests to the AI gateway, so even local inference enters through the governed front door rather than reaching model endpoints directly. The masking is detection-and-redaction at the proxy, so residual false-negative leakage is the control's main failure mode and should be monitored.

Tier 2 is the cost tier: a lightweight classifier selects the cheapest model that can handle the request. The classifier itself adds latency and can misroute, so this tier is a measured trade-off rather than free capacity. The system needs a fallback that escalates to a more capable model on low-confidence or low-quality signals.

Tier 3 is the efficiency tier: once the model pool is fixed, llm-d's endpoint picker maximizes KV-cache hits and balances load across vLLM workers. Red Hat's integration work describes the split the same way: the semantic router picks the model, llm-d schedules replicas within the pool.

Two properties make this path operable, and they're the same two demands you would place on any request path in production. It is traceable end to end when instrumented: OpenTelemetry follows a request across all three tiers, and MLflow Tracing records tool calls, prompts and outputs, latencies and costs. MLflow Tracing itself is production-ready; the OpenTelemetry GenAI semantic conventions it emits are still in development, so plan for attribute changes rather than interface rewrites (see Table 4). And it's decoupled at every hop: every hop speaks OpenAI-compatible or OpenResponses APIs, so the model behind the path can change without the agent noticing.

Job placement

A common design mistake we see is assuming the skills execute inside the agent pod. This habit carries over from library-based development, where importing a capability means hosting its compute resources. In this blueprint, the skill components in Figure 1 represent definitions and thin clients: instruction files and API bindings rather than AI workloads. The distinction keeps the agent pod cheap and the blast radius small. Heavy compute tasks follow one of three patterns, and only pattern 1 runs in the pod: execute in place, call a service, or dispatch a job (Table 3).

Pattern 2 serves as the default model at scale. For this reason, Figure 1 illustrates skill backends as a location-agnostic column outside the platform bars. The optimization service can be cuOpt on a GPU node or OR-Tools on CPU; the agent's client and the gateway policy do not change. Because authorization is endpoint-specific and claims-based, relocating or swapping a backend is a policy change rather than an architecture change. Pattern 3 needs the most attention in security review: an agent that can create Jobs can create arbitrary workloads, so its RBAC must be scoped tightly.

Getting this split right also sets the economics. Agent pods stay resource-efficient, dense, long-lived, and CPU-only in the common case, with the in-pod GPU of pattern 1 as the deliberate exception, while expensive accelerators sit in pooled backends that scale independently. The fleet then scales on two independent axes: agent count on cheap cores, accelerator capacity on utilization, and neither purchase forces the other.

To see the blueprint in action, follow one request end to end; every component named so far makes exactly one appearance. 

A field-service agent receives a dispatch-optimization goal and initializes inside its sandboxed pod. The process then unfolds across the layers:

• The harness calls the optimization skill; the MCP Gateway checks the agent's token claims and forwards the call to the cuOpt or OR-Tools backend.
• The planning prompt carries customer data, so tier 1 keeps it on in-cluster models; a follow-up summarization request is low-risk, so tier 2 routes it to a small model and tier 3 picks the replica with the warmest cache.
• Every tool call, authorization decision, latency, and cost lands in the OpenTelemetry and MLflow trail. 

At no point does the architecture depend on the model performing predictably; the path maintains the same shape even against a hostile model.

Security model

The distinctive threat with a long-running agent is that the workload itself can be turned against you. A prompt-injection attack can redirect its goals, a poisoned tool result can corrupt its memory, and a compromised skill dependency can act with everything the agent is allowed to reach. The design response is to assume the model's behavior cannot be trusted and to route every consequential action through enforcement the model cannot influence.

Figure 4 shows how the architecture layers that enforcement: three nested isolation rings around the agent process, and four cross-cutting controls that apply across all of them. The rings nest deliberately: each one assumes the ring inside it has already failed.

• Ring 1 (process): The sandbox supervisor enforces per-binary, per-path, per-endpoint policy via kernel primitives (seccomp, Landlock, network namespaces). The agent cannot override this policy even if compromised because the policy lives outside the process it constrains.
• Ring 2 (pod): Kubernetes pod security standards, SELinux, AppArmor, NetworkPolicy, and resource quotas bound what the pod can reach and consume. Optionally, Kata Containers add a microVM boundary per sandbox.
• Ring 3 (hardware): Confidential containers extend the trust boundary into silicon, protecting an agent's memory at runtime even if a neighboring workload or the node itself is breached, with attestation proving the sandbox runs the code it claims before any secret is released. This ring is the precondition for multi-tenant agent designs. Confidential computing on GPUs is now generally available on current accelerators but still carries runtime overhead, concentrated in host-to-device transfers, so treat it as the ring you plan for rather than the one you start with.

Four controls cut across all three rings, and they carry the same names in Figure 4:

• Workload identity: SPIFFE SVIDs replace static keys, so a leaked workspace contains nothing durable to steal.
• Tool authorization: The MCP Gateway authorizes every tool call from token claims, never from prompt content.
• Inference guardrails: Input and output screening at the inference boundary (TrustyAI Guardrails Orchestrator, NeMo Guardrails).
• Pre-production testing: Adversarial scanning in CI/CD pipelines catches jailbreak and injection weaknesses before an agent is promoted.

The rings and the controls answer different questions. The rings limit what a compromised agent can do; the controls reduce the chance it gets compromised and remove the rewards if it does. Containment economics follow from the single-sandbox, single-pod, and single-identity mapping: revoking, throttling, or terminating a misbehaving agent is a single targeted operation instead of an incident-wide hunt. The MLflow/OpenTelemetry trail turns every enforcement decision into reviewable evidence, and the posture holds unchanged at the edge as in the data center.

Vendor gravity and escape hatches

Vendor gravity in this stack lives in the libraries and the silicon, one layer below where most lock-in audits look. The license layer is clean: several core building blocks are open source under permissive terms (OpenShell and NemoClaw are Apache-2.0), so no single vendor holds exclusive rights to the architecture, while commercial packaging, NIM profiles, CUDA libraries, and certified support remain vendor-specific. What binds you, when something does, is an unported library or a proprietary interconnect. Here is our layer-by-layer audit:

• Interface-portable: The model layer is any open model behind the same API. The agent blueprint is YAML, portable by construction. The sandbox runtime enforces policies with stock Linux primitives, and Kata Containers plus the agent-sandbox SIG covers the same three rings on any conformant cluster. The whole inference plane is portable: semantic routing and llm-d are vendor-neutral, and vLLM runs on CUDA, ROCm, XPU, and CPU directly, with TPU, Gaudi, and other accelerators supported via hardware plug-ins. The serving architecture therefore survives a full silicon swap untouched at the interface level. Achieving operational parity in performance, tuning, and accelerator coverage is engineering work you should plan for.
• Sticky: The CUDA-X libraries are the real gravity: cuDF, cuOpt, and PhysicsNeMo are CUDA underneath, and NIM containers run only on NVIDIA silicon. Substitutes exist with trade-offs: Polars, DuckDB or Spark for DataFrames, OR-Tools or HiGHS for optimization, and JAX-based surrogates for physics. What the architecture controls is the price of the swap: skill backends sit behind neutral REST/MCP interfaces, so replacing cuOpt with OR-Tools is an endpoint and policy change rather than a redesign.

The rule we enforce throughout the blueprint: every vendor component sits behind a neutral interface. That means OpenAI-compatible and OpenResponses for models, MCP for tools, REST for skill backends, the agent-sandbox API for isolation, and SPIFFE for identity. Under that rule, vendors compete on price/performance per component instead of owning your architecture, and procurement decisions stay reversible for the lifetime of the deployment. NVIDIA then becomes a per-component choice rather than a structural dependency.

Maturity and open items

Not everything in Figure 1 is equally ready. In Table 4, we separate the layers you can run in production today from the layers to treat as preview, where interfaces will still change. The reading rule: build your base decisions on the mature column, adopt from the preview column behind pinned versions and feature gates, and revisit at each release.

And a blueprint earns trust by stating what it does not solve yet. Note that each of the following open items belongs to a different discipline, which is itself the message: closing them takes products, standards bodies, researchers, and operators, and no single vendor closes the list.

• Human approval workflows for high-risk actions still need product-level patterns.
• Multi-agent coordination standards are converging (A2A reached 1.0) but interoperability in practice is not settled.
• Agent memory poisoning remains an open research and engineering problem.
• Sandbox policy authoring is operationally complex at fleet scale.
• Cost modeling for long-running agents needs real workload data.
• Confidential computing protects memory at runtime; it does not stop prompt-layer or tool-layer compromise.

Closing notes

In this article we walked the agentic stack inside out. We defined the agent as model, harness, and sandbox runtime, and laid the solution out in Figure 1 as interchangeable components behind open interfaces. 

Then we followed it through delivery: an agent packaged as one sandboxed pod, or delegating its loop to a shared runtime where the workload profile fits, governed by workload identity and claims-checked tool calls, served through three routing tiers to vLLM workers, with skill compute placed in governed backends and a defense-in-depth security model wrapping it all. 

The result is a blueprint you can begin assembling today from open source parts, treating several layers as preview, with vendor choices that stay reversible at the interfaces you kept neutral. Managed sandbox services offer parts of this blueprint as SaaS; what the open architecture adds is that the isolation, identity, and governance layers stay on infrastructure you control, with interfaces you can take to any conformant cluster. We verified the component landscape in this article as of July 2026; in a stack where several layers are pre-1.0, check the linked projects for current status before you build.

The agentic stack is beginning to look like the microservices stack one abstraction up: blueprints instead of charts, sandboxes instead of containers, inference routing instead of service mesh, skills instead of client libraries, and share d loop runtimes instead of application servers. The same disciplines apply: least privilege, declarative policy, separate control and data planes, pooled expensive resources behind schedulers, and keeping the interfaces under your own control. The difference is that this workload makes its own decisions for days at a time. The stack above is what it takes to let it do that safely, observably, portably, and at a cost per task you can measure.

Where to go next

Architecture arguments are easier to judge with running code, so we are rebuilding our Telecom AI Experiments workloads on this blueprint at agent-school; the 101 NOC Assistant runs offline on a laptop in two commands. To run the blueprint on your own infrastructure, we recommend exploring Red Hat AI, Red Hat OpenShift AI, and Red Hat AI Inference Server.