The State Access Dilemma in Flutter

Executive Summary: For more than six years, Flutter developers have wrestled with how to cleanly read state from the widget tree. Teams were forced to choose between the indentation tax of the "Builder Pyramid" (BlocBuilder nesting) and BuildContext extensions (context.watch, context.select) that carried subtle whole-tree rebuild traps or heavy closure boilerplate. By establishing an elegant 1:1 architectural symmetry between state containers and the widget tree—introducing context.value, context.state, and zero-closure provider tearoffs—BlocSignal eliminates the ceremony. Here is why Flutter state consumption should have always worked this way.

The Friction of Reading State

Every Flutter developer knows the feeling of writing a clean business logic component, only to watch the presentation layer devolve into nested boilerplate:

// The classic Flutter BLoC indentation tax:
class CartSummaryCard extends StatelessWidget {
  const CartSummaryCard({super.key});

  @override
  Widget build(BuildContext context) {
    return BlocBuilder<UserCubit, UserState>(
      builder: (context, userState) {
        return BlocBuilder<CartCubit, CartState>(
          builder: (context, cartState) {
            final discount = userState.isVip ? cartState.subtotal * 0.20 : 0.0;
            final total = cartState.subtotal - discount;

            return Card(
              child: Padding(
                padding: const EdgeInsets.all(16),
                child: Text('Total: \$${total.toStringAsFixed(2)}'),
              ),
            );
          },
        );
      },
    );
  }
}

What is conceptually a simple read of two state values turns into two layers of widget indentation, two anonymous builder closures, and two context shadowing levels.

To avoid this "Builder Pyramid," the community turned to BuildContext extensions. But as teams adopted context.watch and context.select, they encountered an entirely new class of performance pitfalls and developer confusion.

1. The Three Sins of Context Reading in Classic BLoC

To appreciate the modern solution, we must examine the three distinct pain points that have plagued contextual state consumption in Flutter.

Sin 1: The Indentation and Rebuild Tax of BlocBuilder

BlocBuilder works by inserting an internal StatefulWidget into the element tree that subscribes to the BLoC's underlying Dart Stream. While reliable, it forces every state-dependent piece of UI into an explicit builder closure.

When a screen depends on multiple state containers (for example, user authentication, shopping cart items, theme preferences, and localized settings), nesting builders produces severe "pyramid of doom" indentation. Refactoring a widget to depend on one additional piece of state requires wrapping large chunks of widget hierarchy, causing noisy git diffs and fragile layout structures.

Sin 2: The Treacherous context.watch Mental Model Trap

To escape BlocBuilder, classic flutter_bloc introduced context.watch<B>(). But context.watch introduces a dangerous performance trap: it rebuilds the entire enclosing widget whenever any state emits from the BLoC.

@override
Widget build(BuildContext context) {
  // ⚠️ TRAP: Subscribes the ENTIRE widget to every CartState emission:
  final cart = context.watch<CartCubit>().state;

  return Scaffold(
    appBar: AppBar(title: const Text('Store')),
    body: Column(
      children: [
        const HeavyDashboardHeader(), // Rebuilds unnecessarily!
        const PromotionalBanner(),    // Rebuilds unnecessarily!
        Text('Cart Items: ${cart.items.length}'),
        const ProductListView(),       // Rebuilds unnecessarily!
      ],
    ),
  );
}

Even if you only needed cart.items.length in a single Text widget, the entire Scaffold, its AppBar, and every heavy child widget rebuild on every emission.

The Signal Architecture "Scar"

When reactive signal engines emerged, this trap became even more confusing. In bloc_signals_flutter, state updates propagate synchronously via fine-grained signals rather than Stream-backed InheritedWidget mutations. Consequently, context.watch<B>() was implemented to track container instance swapping only (ensuring inherited dependencies update when a parent widget swaps container instances), not state emissions.

Developers coming from classic flutter_bloc who wrote:

final count = context.watch<CounterCubit>().stateValue;

walked straight into an architectural trap: their UI never rebuilt on state emissions. Because context.watch only checks container instance identity (bloc != oldWidget.bloc), state mutations were silently ignored by the widget element.

Sin 3: Closure Fatigue in context.select

To solve whole-widget rebuilds, libraries introduced context.select:

final count = context.select<CounterCubit, int>(
  (cubit) => cubit.stateValue.count,
);

While context.select provides surgical, fine-grained rebuilds, it imposes heavy syntax friction. For every single value you want to display, you must provide:

1. The container type generic (CounterCubit).
2. The return type generic (int).
3. An anonymous extraction lambda (cubit) => cubit.stateValue.count.

When writing modern Flutter code, writing (c) => c.stateValue dozens of times across UI widgets creates undeniable closure fatigue.

2. The Architectural Symmetry: Container vs. Context

The solution in BlocSignal is rooted in a fundamental architectural principle: 1:1 Conceptual Symmetry.

In modern reactive architectures, you interact with state in two distinct forms:

1. The Reactive Signal (ReadonlySignal<S>): A push-pull reactive primitive designed for dependency tracking, computed derived values, and observable subscriptions.
2. The Unwrapped Value (S): The raw immutable domain object ready for direct display or evaluation.

Historically, state management libraries mixed and matched inconsistent naming conventions across the container and the widget context. BlocSignal creates strict, predictable symmetry across both levels:

┌────────────────────────────────────────────────────────────────────────┐
│                        ARCHITECTURAL SYMMETRY                          │
├───────────────────┬──────────────────────────┬─────────────────────────┤
│ Scope             │ Reactive Signal          │ Unwrapped State Value   │
├───────────────────┼──────────────────────────┼─────────────────────────┤
│ Container Level   │ cubit.state              │ cubit.value             │
│ Widget Context    │ context.state<B, S>()    │ context.value<B, S>()   │
│ Web / Jaspr       │ context.state<B, S>()    │ context.value<B, S>()   │
└───────────────────┴──────────────────────────┴─────────────────────────┘

Notice the simplicity:

• On your state container:
  • cubit.state returns ReadonlySignal<S>.
  • cubit.value returns raw S (with cubit.stateValue preserved as a permanent alias).
• On your widget BuildContext:
  • context.state<B, S>() returns ReadonlySignal<S> (for reactive signal graph composition).
  • context.value<B, S>() returns raw S (and subscribes the widget element to rebuilds).

3. Deep Dive: context.value<B, S>()

context.value<B, S>() provides single-line, zero-closure state subscription directly inside your widget's build() method.

class CounterDisplay extends StatelessWidget {
  const CounterDisplay({super.key});

  @override
  Widget build(BuildContext context) {
    // 1-line read that automatically subscribes this element to updates:
    final count = context.value<CounterCubit, int>();

    return Text('Count: $count');
  }
}

How It Works Under the Hood

Under the hood, context.value<B, S>() delegates directly to BlocSignal's optimized select engine:

extension BlocSignalProviderExtension on BuildContext {
  S value<T extends BlocSignalBase<S>, S>() {
    return select<T, S>((bloc) => bloc.value);
  }
}

Because it routes through select:

1. It looks up T via BlocSignalProvider.of<T>(this, listen: true).
2. It attaches a fine-grained element subscription that triggers element.markNeedsBuild() only when bloc.value emits a new state.
3. It eliminates the need to pass an extraction closure (b) => b.value.

Scoped Micro-Rebuilds with Standard Flutter Builder

Because context.value binds to the calling BuildContext element, you can scope rebuild boundaries using standard, built-in Flutter widgets like Builder without ever importing a specialized builder component:

class ProductCheckoutScreen extends StatelessWidget {
  const ProductCheckoutScreen({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Checkout')),
      body: Column(
        children: [
          // Heavy static widgets that NEVER rebuild:
          const CheckoutBanner(),
          const ShippingAddressCard(),

          // Surgical micro-rebuild scope:
          Builder(
            builder: (scopedContext) {
              // Only this inner Builder element rebuilds when Cart updates:
              final cart = scopedContext.value<CartCubit, CartState>();
              return Text('Total: \$${cart.subtotal.toStringAsFixed(2)}');
            },
          ),
        ],
      ),
    );
  }
}

You get surgical rebuild boundaries, zero third-party builder nesting, and 100% native Flutter element semantics.

4. Deep Dive: context.state<B, S>()

What if you want to look up a state container from the widget tree, but you do not want your widget element to rebuild when that state changes?

For example, what if you are constructing a derived signal graph using computed() or feeding multiple state containers into a SignalBuilder?

That is the exact role of context.state<B, S>().

class DiscountBadge extends StatelessWidget {
  const DiscountBadge({super.key});

  @override
  Widget build(BuildContext context) {
    // 1. Look up reactive signals WITHOUT subscribing this widget element:
    final cartSignal = context.state<CartCubit, CartState>();
    final userSignal = context.state<UserCubit, UserState>();

    // 2. Compose reactively inside SignalBuilder:
    return SignalBuilder(
      builder: (context) {
        final isVip = userSignal.value.isVip;
        final subtotal = cartSignal.value.subtotal;
        final discount = isVip ? subtotal * 0.20 : 0.0;

        return Text('VIP Savings: \$${discount.toStringAsFixed(2)}');
      },
    );
  }
}

Why Not Just Use context.read<B>().state?

A common question from architects is: Why can't I just write context.read<CartCubit>().state?

The answer lies in one of the most insidious bugs in Flutter provider trees: The Zombie Subscription Trap.

Imagine an application where an ancestor widget replaces or swaps the provided container instance. For example:

• A user selects a different workspace or account, causing an ancestor BlocSignalProvider.value(value: newCubit) to swap instances.
• A test harness or modal flow injects a fresh container into a subtree.

If your widget looked up the container with context.read<CartCubit>().state:

• context.read does not register an inherited widget dependency.
• When the ancestor provider swaps the cubit, your widget is never notified.
• Your reactive signal effects and computed() properties remain permanently attached to the discarded, dead cubit instance, leaking memory and failing to reflect updates from the new container.

context.state<B, S>() solves this permanently:

ReadonlySignal<S> state<T extends BlocSignalBase<S>, S>() {
  return BlocSignalProvider.of<T>(this, listen: true).state;
}

Because _BlocSignalProviderInherited.updateShouldNotify checks bloc != oldWidget.bloc:

• During regular state emissions, updateShouldNotify returns false. listen: true costs exactly zero extra rebuilds.
• If an ancestor provider replaces the container instance with a new one, updateShouldNotify returns true, immediately triggering a rebind of your signal references without leaving zombie subscriptions.

5. Multi-Container Reactive Composition Without Multi-Bloc Builders

In classic flutter_bloc, composing multiple state machines in the UI required either:

1. MultiBlocBuilder or heavily indented nested builders.
2. Creating an artificial "coordinator BLoC" whose only job was subscribing to both streams and emitting a merged state.

With context.state and signals, multi-container composition becomes effortless.

Let's compare the two approaches side-by-side:

The Classic BLoC Approach (Ceremonial Indentation)

// ❌ Classic flutter_bloc: Nested builders or MultiBlocBuilder
Widget build(BuildContext context) {
  return MultiBlocListener(
    listeners: [
      BlocListener<CartBloc, CartState>(listener: (context, state) => ...),
      BlocListener<UserBloc, UserState>(listener: (context, state) => ...),
    ],
    child: BlocBuilder<CartBloc, CartState>(
      builder: (context, cartState) {
        return BlocBuilder<UserBloc, UserState>(
          builder: (context, userState) {
            final total = calculateDiscount(cartState, userState);
            return Text('Total: \$total');
          },
        );
      },
    ),
  );
}

The Modern BlocSignal Approach (Push-Pull Reactivity)

// ✅ Modern BlocSignal: Pure reactive signal composition
Widget build(BuildContext context) {
  final cart = context.state<CartCubit, CartState>();
  final user = context.state<UserCubit, UserState>();

  return SignalBuilder(
    builder: (context) {
      final subtotal = cart.value.subtotal;
      final discount = user.value.isVip ? (subtotal * 0.20) : 0.0;
      return Text('Total: \$${(subtotal - discount).toStringAsFixed(2)}');
    },
  );
}

Whenever cart emits, SignalBuilder recalculates. Whenever user emits, SignalBuilder recalculates. If neither emits, zero CPU cycles are spent.

No streams, no coordinators, no nested builder widgets.

6. Zero-Closure Provider Tearoffs (Cubit.create)

The ergonomic improvements extend to how state containers are injected into the widget tree.

In Dart, generative constructor tearoffs (for example CounterCubit.new) have a zero-parameter signature: CounterCubit Function().

However, Flutter provider APIs require a factory closure that accepts a BuildContext:

// The traditional lambda tax:
BlocSignalProvider<CounterCubit>(
  create: (context) => CounterCubit(),
  child: const CounterPage(),
)

Typing (context) => MyCubit() across dozens of providers in large applications adds minor but persistent friction.

By defining a dedicated .create named constructor that accepts BuildContext _ and redirects to the default constructor, you unlock zero-closure constructor tearoffs:

class CounterCubit extends CubitSignal<int> {
  CounterCubit() : super(initialState: 0);

  /// Zero-closure provider factory constructor.
  CounterCubit.create(BuildContext _) : this();

  void increment() => emit(value + 1);
}

Now, providing containers in your widget tree is completely lambda-free:

// ✅ Zero-closure constructor tearoffs:
MultiBlocSignalProvider(
  providers: const [
    BlocSignalProvider<CartCubit>(create: CartCubit.create),
    BlocSignalProvider<UserCubit>(create: UserCubit.create),
  ],
  child: const ShoppingApp(),
)

7. Hands-On: The Context Ergonomics Showcase

To demonstrate these patterns in a production-style application, we added a complete, runnable showcase to the BlocSignal monorepo: examples/flutter_context_ergonomics.

examples/flutter_context_ergonomics/
├── lib/
│   └── main.dart            # Complete 3-tab interactive showcase
├── test/
│   └── widget_test.dart     # Comprehensive widget test suite
├── README.md                # Run instructions and pattern tour
└── pubspec.yaml

The sample application features:

1. Interactive Cart & User Cubits: Complete with zero-closure CartCubit.create and UserCubit.create tearoffs.
2. Tab 1 (context.value): Live rebuild counter badges comparing parent widget scopes with surgical Builder micro-rebuilds. Tapping "Add Item" proves that only the inner badge rebuilds, while the outer widget scope build counter remains completely static.
3. Tab 2 (context.state): Real-time multi-cubit discount calculations inside SignalBuilder. Toggling VIP membership or adjusting cart items triggers instant reactive recalculations with zero coordinator boilerplate.
4. Tab 3 (Architecture Matrix): An interactive reference comparing all five BuildContext state access methods in detail.

To run the example locally:

cd examples/flutter_context_ergonomics
flutter run -d chrome # or macos, ios, android

To run the automated test suite:

flutter test

8. Cross-Platform Parity: Jaspr Web & SSR

One of BlocSignal's greatest strengths is that its reactive principles are not confined to Flutter mobile and desktop.

Through bloc_signals_jaspr, the exact same ergonomic extensions are available when building web applications and server-side rendered (SSR) components with Jaspr:

import 'package:bloc_signals_jaspr/bloc_signals_jaspr.dart';
import 'package:jaspr/jaspr.dart';

class WebCartSummary extends StatelessComponent {
  const WebCartSummary({super.key});

  @override
  Iterable<Component> build(BuildContext context) sync* {
    // Exact same API in Jaspr Web:
    final cart = context.value<CartCubit, CartState>();

    yield div([
      h2([Component.text('Cart (${cart.totalCount} items)')]),
      p([Component.text('Subtotal: \$${cart.subtotal}')]),
    ]);
  }
}

Whether you are writing a high-frequency Flutter mobile app or rendering HTML on the web, your mental model, state access methods, and architectural boundaries remain 100% identical.

9. Summary: The Modern State Access Cheat Sheet

When working with BlocSignal in your Flutter presentation layer, use this simple cheat sheet to select the exact method matching your intent:

Conclusion

Frontend state management should make reading state feel natural, predictable, and clean.

By eliminating the ceremony of the Builder Pyramid, solving the context.watch mental model trap, and establishing strict 1:1 symmetry between containers and widget contexts, context.value and context.state make writing reactive Flutter applications a joy.

Give the new ergonomics a try in bloc_signals_flutter: ^1.4.0 and bloc_signals_jaspr: ^1.2.0, explore the runnable showcase, and let us know what you think in the comments!

Amazon Web Services (AWS) has released a new open-source application dubbed Pizza Bot, which gives developers an email-style inbox for managing AI agents that run in the background.

The problem that Pizza Bot is designed to address, ultimately, is that a chat interface is a poor fit for agents whose work continues after the (human) user has clocked off for lunch or bed.

And so Pizza Bot leans on the age-old mechanics of email: completed jobs arrive as unread threads, while anything that needs a human decision is surfaced for action. An Activity panel also exposes jobs handed off to specialist agents, including their tool use and progress.

“Scheduled agents run autonomously in the background and surface updates directly into your inbox for review and triage.”

Writing about the new project in a LinkedIn post on Thursday, co-creator Joseph Dolivo, principal technologist at AWS Startups, notes that Pizza Bot shifts the burden of monitoring agent work firmly away from the user.

“Instead of you having to initiate every conversation or wait on a prompt, scheduled agents run autonomously in the background and surface updates directly into your inbox for review and triage,” he writes.

As if to emphasize that point, in the accompanying blog post for the project’s official launch on Thursday, the creators note that user absence is, in fact, a core tenet of the design brief.

“The interface assumes you are not watching.”

“The interface assumes you are not watching,” they write. “Nothing else we’ve seen starts there, and that one assumption is what buys you pauses that outlast the session that created them, notifications worth acting on, and scheduled work that produces threads instead of logs.”

A community project

Despite its Amazon roots, Pizza Bot is in fact now a standalone community project rather than an AWS service. It lives in its own GitHub organization, separate from Amazon, and comes with no AWS support or service-level agreement — it’s entirely self-hosted.

Pizza Bot itself is a desktop app for macOS, Windows, and Linux, with browser and terminal clients available too. By default, the app starts a local Pizza Bot server on the machine, while developers choose the model behind it — including Anthropic, Amazon Bedrock, Google Gemini, OpenAI, OpenRouter or a local model via Ollama.

It can also be extended through MCP servers and Agent Skills; the bundled browser-automation skill, for example, uses Playwright MCP to navigate and interact with websites.

The server can also run on an always-on host or in a container, letting scheduled agents keep working while the laptop is closed and their threads be picked up later from another device.

Under the hood: LangGraph and ambient agents

Pizza Bot’s agent runtime is built with DeepAgents, LangChain’s open-source harness for long-running agent tasks, which itself runs on LangGraph, its runtime for stateful agent execution. The important part in all of this is persistence: LangGraph checkpoints an agent’s state as it works, allowing a run to stop for approval, survive a disconnected client and resume later without starting again from scratch. Pizza Bot stores those checkpoints, along with threads and other application data, locally in SQLite and ordinary files.

It’s worth noting that AWS has its own open source agents SDK, Strands Agents, out since May 2025 — but evidence suggests, including text in this sample repository, that AWS considers Strands as a “lighter-weight alternative to LangGraph for agents that don’t need explicit graph control flow.”

In response to a question posted on LinkedIn by The New Stack, Dolivo says that they very well could have used Strands for Pizza Bot, particularly as Strands supports TypeScript and workflows now. But they ultimately went for LangGraph “due to the maturity of the tooling and breadth of the exosystem,” he explains.

“It’s also more familiar to many developers, and we wanted to reduce friction for community adoption since we knew we’d be open-sourcing it,” he adds.

Pizza Bot is also fairly close to an idea LangChain introduced way back in January 2025, when CEO Harrison Chase introduced the term “ambient agents” for agents that could respond to events, work concurrently and involve a human only when needed. LangChain’s reference implementation was an email assistant built on LangGraph. It also developed what it called an “Agent Inbox”: a standalone interface inspired by email and customer-support software for keeping track of open interactions between people and background agents.

From ‘JoeBot’ to Pizza Bot

The genesis of Pizza Bot can be traced back to April 2025, when Dolivo kicked off what he calls a “side-of-desk passion project” dubbed “JoeBot” that automated repetitive CRM logging. He later teamed up with colleague Igor Fil to turn that script into Pizza Bot, an MCP server that could execute parameterized, deterministic “recipes” across its internal systems.

The appeal soon spread beyond the engineers building it into less-technical domains. As the project evolved, Dolivo says it would eventually grow to more than 30 contributors and over 2,000 users inside Amazon, and so Pizza Bot needed a front end that people could open and use.

“An MCP server requires an MCP client, and expecting non-technical users to work out of an IDE or terminal was never going to cut it,” Dolivo adds. “We had to meet people where they actually work and own the experience end to end.”

The result was the version of Pizza Bot released this week: a desktop app built around an inbox rather than a terminal.

The post AWS open-sources Pizza Bot: email-style inbox for background AI agents appeared first on The New Stack.

Security Engineering is all about tradeoffs:

• Security vs. Privacy
• Security vs. Performance
• Security vs. Compatibility
• Security vs. Usability

Web Browsers attempt to achieve an absolutely bananas goal: Allow safe execution of untrusted content on a user’s device.

Browsers are a huge vector for compromise of users’ devices and personal information, owing to the power and complexity. Much of the vulnerability induced by browsers occur where tradeoffs were either made poorly initially, or where the tradeoff would be made differently knowing what we know now.

So, what should we do? Here’s a modest list of proposals, many of which could be achieved in less than one dev day:

1. Disallow random websites from going fullscreen without permission
2. Allow simple Enterprise control of what types of files are allowed to download — current controls are comically underpowered. (https://issues.chromium.org/issues/40265750)
3. Block download UI launch of high-risk file types that the OS has inexplicably failed to secure
4. Introduce a pre-fetch security check to allow security software to block malicious requests (similar to this)
5. Call AMSI to detect malicious content copied to the clipboard (https://issues.chromium.org/issues/440381280)
6. Call AMSI when installing a new browser extension or restarting the browser to allow local security software insight of what code can impact the user’s browsing experience
7. Stop supporting UserInfo in URLs or introduce a warning
8. Disallow user-navigation to javascript: URLs or introduce a warning (https://issues.chromium.org/issues/559142626)
9. Further restrict notification permissions to prevent scams and spam
10. more to come, I’m sure

In Kubernetes, resource allocation has historically been a static decision made during a Pod's initial scheduling and placement. With the graduation of the core in-Place Pod resize feature to General Availability in v1.35, application developers and cluster operators gained the powerful ability to dynamically adjust CPU and memory allocations of running containers without incurring disruptive restarts or application downtime.

However, in-place resizing introduced a unique resource scheduling gap: if a running Pod requested a resource scale-up that exceeded the host node's allocatable headroom, the Kubelet was forced to mark the request as Deferred. The Pod would remain parked in this state indefinitely, waiting for resources on the node to naturally free up.

To bridge this scheduling gap, Kubernetes v1.37 introduces scheduler preemption for in-place Pod resize (Alpha), behind the InPlacePodVerticalScalingSchedulerPreemption feature gate. This feature allows the Kubernetes scheduler to actively free up capacity on a fully-utilized node by preempting lower-priority workloads, enabling the pending in-place resizes of critical, higher-priority applications to succeed.

The "deferred" resize challenge

To understand why this preemption mechanism is needed, it is helpful to look at how Kubernetes handles running Pod resizing. When a user or controller (such as the Vertical Pod Autoscaler) updates the resource requests of an active container, the Kubelet evaluates whether the underlying node has enough spare allocatable capacity to fulfill the increase.

If the node's resources are fully utilized and cannot satisfy the new limits, the Kubelet sets the container's resizeStatus (reported in the Pod's status.containerStatuses[]) to Deferred. Unlike an Infeasible resize request (which is immediately rejected because it exceeds physical machine boundaries, namespace limit ranges, or admission quotas) a Deferred status indicates that the request is valid but is temporarily unable to be actuated, waiting until node capacity becomes available.

Before the introduction of this preemption mechanism, a Pod's in-place resize scale-up request could become permanently blocked if the node was heavily utilized. Even when a critical application (such as an in-memory database or a real-time web server) required more memory to prevent an imminent out-of-memory (OOM) crash, and the node lacked free capacity, the resize remained Deferred.

In this scenario, cluster administrators had limited choices:

1. Manually evict lower-priority Pods from the node to clear resource headroom.
2. Rely on the cluster autoscaler to eventually spin up a larger node and reschedule the Pod. However, this is an operation that is highly disruptive and violates the core "no restart" value proposition of in-place scaling.
3. Rely on a custom autoscaling solution, for example a cluster autoscaler that can trigger dynamic node resizing operations itself.

Because the kube-scheduler was unaware of deferred resizes on running Pods, it could not leverage standard priority-based preemption to evict lower-priority workloads and make room for the higher-priority running Pod's resource growth.

Why this matters

In production Kubernetes environments, cluster administrators strive to maximize resource utilization and efficiency. A common strategy is to bin-pack unused capacity on not-yet-full nodes with lower-priority workloads, such as batch jobs, background data processing, or best-effort tasks.

Without scheduler preemption for in-place resizing, this created a major operational dilemma. If lower-priority workloads consumed the remaining headroom on a node, higher-priority applications running on that same node would become blocked (Deferred) when they needed to scale up to handle sudden traffic surges or memory spikes. Operators were forced to choose between running low-utilization clusters with idle buffer capacity or risking that critical workloads could not resize when needed.

With scheduler preemption for in-place Pod resize, you can confidently bin-pack unused space across your clusters with lower-priority workloads without worrying about them degrading higher-priority Pods or blocking their scale-up requests. If a high-priority workload requires an in-place resize that exceeds available node capacity, the scheduler automatically preempts the lower-priority Pods to clear headroom. You achieve high cluster utilization and cost efficiency while preserving the responsiveness and reliability of critical services.

Architectural mechanics: How it works

Scheduler preemption for in-place Pod resize integrates directly into the core scheduling cycle to coordinate resources dynamically and safely.

Centralized scheduler tracking

The kube-scheduler monitors the cluster for running Pods with a Deferred resize status condition. Normally, Pods with spec.nodeName populated are considered successfully placed and bypass the active scheduling queue. Under this feature gate, the scheduler intercepts Pods carrying the Deferred condition, permitting them to remain in active scheduling evaluations specifically to trigger preemption. The scheduler maintains continuous tracking of these Pods until the Kubelet successfully completes the resize actuation.

Single-node preemption boundary

Unlike placement preemption, which evaluates all nodes in a cluster to find the best scheduling fit, preemption for in-place resizing is strictly localized to the Pod's currently assigned node. The scheduler identifies eligible lower-priority "victim" Pods on the same host and initiates their graceful eviction, freeing up local capacity. Preemption is strictly scoped to the same node where the deferred Pod is running; if a node cannot accommodate the resize even after evicting all eligible lower-priority workloads, the resize remains in the Deferred state.

Resource reservation safety

To prevent scheduling races and double-allocation, the scheduler treats resources requested for a resize as already consumed. This enables the Kubelet to actuate the resize once the preemption takes effect.

Separation of concerns & critical admission

When a node is under resource pressure, the Kubelet includes a local mechanism known as the critical Pod admission handler. During initial Pod admission, if a critical system Pod arrives on a node that lacks spare capacity, this local handler can directly evict lower-priority Pods on that node to guarantee admission for the critical workload.

A significant architectural benefit of this new feature is the strict separation of concerns between the Kubelet and the scheduler. Under the InPlacePodVerticalScalingSchedulerPreemption feature gate, the Kubelet's critical Pod admission handler does not perform local preemption checks or trigger local evictions for in-place resizing operations. Instead, the Kubelet defers the request and delegates the preemption decision entirely to the scheduler. This guarantees that a single, centralized orchestrator manages all resize-related preemption logic, respecting global priorities, Pod disruption budgets (PDBs), and graceful termination policies.

Managing competing updates & races

If a competing, higher-priority resize request is submitted for another running Pod on the same node during an active preemption cycle, the Kubelet prioritizes the higher-priority request. The scheduler is designed to observe these updates and will dynamically trigger a new round of preemption if more capacity is required to fulfill the new state.

Node-level preemption configuration

Administrators and automated controllers (such as a cluster autoscaler) can disable preemption specifically for in-place resizes on particular nodes. This is configured using the new spec.podPreemptionPolicy field in the Node Spec:

apiVersion: v1
kind: Node
metadata:
 name: batch-workload-node
spec:
 podPreemptionPolicy:
 disableResizePreemption:
 - "cluster-autoscaler.kubernetes.io/disable-preemption"
 - "operator.example.com/policy-override"

An example use case for this policy is when a controller would prefer to size down other pods or dynamically adjust the node capacity itself when possible, only enabling scheduler preemption as a last resort.

Try it out!

To utilize scheduler preemption for in-place Pod resize:

• Your cluster must be running Kubernetes v1.37 or later across both the control plane and all worker nodes.
• The InPlacePodVerticalScalingSchedulerPreemption feature gate must be enabled across all control plane components (kube-apiserver, kube-scheduler) and the kubelet.

Mini-tutorial: Observe resize preemption in action

To see this feature in action locally, you can test scheduler preemption on a single-node kind cluster with constrained CPU headroom.

1. Create a kind cluster with scheduler resize preemption enabled

Create a kind cluster configuration file named kind-config.yaml with the InPlacePodVerticalScalingSchedulerPreemption feature gate enabled:

# kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
 InPlacePodVerticalScalingSchedulerPreemption: true

Create the cluster using this configuration, passing the --image flag to ensure the cluster is running Kubernetes v1.37 (or later):

kind create cluster --config kind-config.yaml --image kindest/node:v1.37.0

Once your cluster is ready, inspect the node to check how many allocatable CPU cores it has:

kubectl get nodes -o custom-columns=NAME:.metadata.name,ALLOCATABLE_CPU:.status.allocatable.cpu

In a standard local kind environment, the output shows 8 allocatable CPU cores:

NAME ALLOCATABLE_CPU
kind-control-plane 8

2. Create PriorityClasses and deploy Pods

Create two PriorityClasses and deploy a low-priority Pod (requesting 3 CPU) alongside a high-priority Pod (requesting 4 CPU). Together, these workloads consume 7 of the 8 available CPU cores, leaving 1 CPU of free allocatable headroom on the node.

# preemption-demo.yaml
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
 name: high-priority
value: 1000000
globalDefault: false
description: "High priority workload"
---
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
 name: low-priority
value: 1000
globalDefault: false
description: "Low priority workload"
---
apiVersion: v1
kind: Pod
metadata:
 name: low-priority-pod
spec:
 priorityClassName: low-priority
 containers:
 - name: worker
 image: nginx
 resources:
 requests:
 cpu: "3"
 memory: "500Mi"
 limits:
 cpu: "3"
 memory: "500Mi"
---
apiVersion: v1
kind: Pod
metadata:
 name: high-priority-pod
spec:
 priorityClassName: high-priority
 containers:
 - name: app
 image: nginx
 resources:
 requests:
 cpu: "4"
 memory: "1Gi"
 limits:
 cpu: "4"
 memory: "1Gi"

Save this manifest to preemption-demo.yaml and apply it:

kubectl apply -f preemption-demo.yaml

Wait until both Pods are running on the node:

kubectl get pods

Output:

NAME READY STATUS RESTARTS AGE
high-priority-pod 1/1 Running 0 9s
low-priority-pod 1/1 Running 0 9s

3. Request an in-place scale-up

Patch the high-priority Pod to increase its CPU request from 4 to 6 (+2 CPU delta). Because only 1 CPU of headroom is free on the node, this resize request exceeds remaining allocatable capacity:

kubectl patch pod high-priority-pod --subresource resize --patch \
 '{"spec":{"containers":[{"name":"app", "resources":{"requests":{"cpu":"6"}, "limits":{"cpu":"6"}}}]}}'

4. Inspect the preemption event on the low-priority Pod

With InPlacePodVerticalScalingSchedulerPreemption enabled, the scheduler intercepts the Deferred resize condition on high-priority-pod and targets low-priority-pod for preemption.

To verify that the scheduler actively preempted the low-priority Pod, inspect its events:

kubectl get events --field-selector involvedObject.name=low-priority-pod

In the event stream (or via kubectl describe pod low-priority-pod), you will see a Preempted event emitted by the scheduler:

LAST SEEN TYPE REASON OBJECT MESSAGE
5s Normal Preempted pod/low-priority-pod Preempted by pod 97dba925-6b5f-4e2f-99f9-d51c30016586 on node kind-control-plane
5s Normal Killing pod/low-priority-pod Stopping container worker

5. Trace the resize event lifecycle on the high-priority Pod

Next, inspect the event history on high-priority-pod to observe how the resize progressed from being deferred to successfully completed:

kubectl get events --field-selector involvedObject.name=high-priority-pod

You will observe a sequence of events as the Kubelet coordinates with the scheduler:

LAST SEEN TYPE REASON OBJECT MESSAGE
33s Warning ResizeDeferred pod/high-priority-pod Pod resize OutOfcpu: {"containers":[{"name":"app","resources":{"limits":{"cpu":"6","memory":"1Gi"},"requests":{"cpu":"6","memory":"1Gi"}}}],"generation":2,"error":"Node didn't have enough resource: cpu, requested: 6000, used: 3950, capacity: 8000"}
32s Normal ResizeStarted pod/high-priority-pod Pod resize started: {"containers":[{"name":"app","resources":{"limits":{"cpu":"6","memory":"1Gi"},"requests":{"cpu":"6","memory":"1Gi"}}}],"generation":2}
32s Normal ResizeCompleted pod/high-priority-pod Pod resize completed: {"containers":[{"name":"app","resources":{"limits":{"cpu":"6","memory":"1Gi"},"requests":{"cpu":"6","memory":"1Gi"}}}],"generation":2}

1. ResizeDeferred: The Kubelet initially marks the resize request as deferred (Warning) due to insufficient CPU headroom on the node (OutOfcpu).
2. ResizeStarted: Once the scheduler preempts low-priority-pod and capacity is released, the Kubelet accepts the new allocation and begins actuating the resize.
3. ResizeCompleted: The Kubelet successfully updates container cgroup limits via the container runtime without restarting the Pod.

Finally, verify that the allocated CPU on the container reflects the new request (appending {"\n"} to the JSONPath query ensures a trailing newline in your terminal):

kubectl get pod high-priority-pod -o jsonpath='{.status.containerStatuses[0].allocatedResources.cpu}{"\n"}'

Output:

6

This confirms that the in-place resize succeeded.

Getting involved

This feature represents a major step forward for resource scheduling, bringing enterprise-grade density control and workload prioritization to dynamic resource scaling. We invite cluster operators, platform architects, and developers to enable the InPlacePodVerticalScalingSchedulerPreemption feature gate in their testing environments and share feedback.

If you want to share your experience with this feature, please get in touch with the community via SIG Scheduling or SIG Node channels!

This blog post is the fourth and final installment of The Economics of Agent Optimization, which shares the strategies, capabilities, and proof points that can help you optimize agent costs and run AI as a managed investment system on Microsoft Foundry. The first post set out the three decisions that systems rest on, the second post took the request at runtime, and the third post took the workflow over time. This post takes the decision that never stops running: governing the spend.

AI agents are moving from isolated pilots into an enterprise estate. They work across teams, connect to data and tools, and make decisions with varying degrees of autonomy. For IT leaders, that creates a broader operating question: how do you govern a agentic system that can grow and act faster than traditional applications?

AI agent governance starts with knowing which agents exist, who owns them, what they can access, and which policies apply. It is often discussed in terms of security, compliance, and lifecycle management. It is also fundamental to cost optimization. Without consistent governance, each team makes its own choices about models, tools, capacity, and limits—and small inefficiencies multiply across every agent and every turn.

Good governance makes consumption visible, attributable, and bounded. IT needs to see which agents and teams are driving usage. Finance needs budgets and cost allocation it can trust, without discovering an unexpected increase after the invoice closes. Developers need controls that can respond at the speed agents run.

That last requirement exposes an important distinction. Traditional cost management tools can track spending and alert on actual or forecasted costs, but they typically operate on billing data rather than in the request path. An agent caught in a retry loop does not wait for the next budget evaluation.

A budget alert is a smoke detector. An agent also needs a circuit breaker. Effective cost governance therefore depends on three things: seeing the spend, bounding it, and proving the return.

See the spend where it starts

AI costs become difficult to manage when they arrive as one aggregate number. One deployment may serve several agents; one agent may use several models and tools; and one outcome may require many turns. By the time that appears on an invoice, the business context has disappeared.

Cost management capabilities in Foundry brings that context closer to the systems creating it. Teams can see estimated costs across projects, inspect cost and token usage for individual agents, and monitor model costs. These estimates support operating decisions; Microsoft Cost Management and invoiced charges remain the system of record for financial reconciliation.

Foundry also supports project-level cost attribution. Every Foundry project is automatically associated with a project tag on its underlying usage. FinOps teams can filter Cost Analysis by that tag to allocate spending to the business unit, team, or workload that incurred it. This capability is currently in preview for models sold by Microsoft Azure, including Azure OpenAI.

At the gateway, Azure API Management’s AI Gateway can emit token metrics by API, product, user, subscription, gateway, and backend. Tracing in Foundry captures tool usage, retries, latency, token consumption, and costs for an agent run.

Together, observability signals explain not only how much an agent consumed, but why:

• Traces reveal model calls, tool invocations, retries, latency, and token usage.
• Monitoring surfaces production trends and anomalies.
• Evaluations measure quality, safety, groundedness, and task completion. Run continuously, they give teams evidence to test whether a smaller model still meets their quality bar rather than defaulting to the largest one. Safety evaluators can also flag issues such as prompt injection, sensitive data leakage, and harmful content before they reach production, where remediation can be costly.

Viewed together, these signals help teams understand whether rising costs are driven by customer demand, inefficient agent behavior, quality regressions, or architectural issues.

That context turns cost data into actionable governance. Before teams can set limits or measure ROI, they need to understand how agents behave in production.

Set spend limits at every layer

Visibility tells you where the money went. Limits determine whether it can keep going. There are three layers to the control system, each working at a different scope and speed:

1. Enforce limits in Foundry

With AI Gateway configured, Foundry Control Plane can enforce tokens-per-minute rate limits and total token quotas for model deployments at the project scope. A request that exceeds the rate limit receives a 429 Too Many Requests response. A caller that exhausts its token quota receives a 403 Forbidden response.

Unlike a cost alert, enforcement happens in the request path. Teams can contain one project’s consumption before it monopolizes shared capacity and establish different boundaries for different projects. Quotas can operate over hourly, daily, weekly, monthly, or yearly periods. Teams can configure the Azure API Management-backed gateway and manage its token limits through Foundry Control Plane.

2. Apply policy across models and providers

For controls spanning projects or model providers, the llm-token-limit policy limits consumption per key using a rate, a cumulative quota, or both. The key can represent a subscription, application, team, customer, workload identity, or another business boundary.

AI Gateway applies the same governance model across OpenAI-compatible APIs, the Anthropic Messages API, as well as MCP servers and agent-to-agent APIs. Backend load balancing can prioritize provisioned capacity before spilling over to pay-as-you-go deployments, while circuit breakers can temporarily stop sending requests to a failing or throttled backend.

Like any distributed limit, these controls have boundaries. Counters are maintained independently at each gateway, and concurrent requests can create a small temporary overage because final token consumption is known only after responses return. The goal is to replace unbounded consumption with a predictable operating boundary.

3. Use financial budgets for accountability and escalation

Microsoft Cost Management budgets serve a different purpose from token limits. They use Azure billing data, including actual prices, credits, and purchasing commitments, to give finance and IT an authoritative view of what the organization has spent and is forecast to spend.

Teams can set budget thresholds and notify owners when actual or forecasted costs approach them. They can also connect a budget to an Azure Monitor action group, which can invoke a customer-designed workflow such as opening a ticket, notifying an operations team, or starting a Logic App or automation runbook. Cost anomaly detection provides another warning when spending departs from its historical pattern.

These are valuable accountability and escalation tools, but they are not instant spending caps. They respond to billing data after consumption occurs. Token limits operate earlier, in the path of each model request, where they can reject new calls after a rate limit or quota is reached. Organizations need both: token limits to contain consumption as agents run, and financial budgets to keep owners accountable and prevent finance from being surprised.

Today, these two layers use different units. The platform enforces consumption in tokens, while finance plans and allocates investment in dollars. Because token prices vary by model and offer, a token quota does not translate into one stable dollar amount.

We are actively working to close that gap with future capabilities in Microsoft Foundry and the AI Gateway in Azure API Management that bring dollar-denominated budgets, finer-grained attribution, and policy-driven controls closer to where agents run.

Measure the value the agent creates

Putting a ceiling on consumption solves only half of the governance problem.

While cost controls can help organizations manage spending, they do not answer a more important question: is the agent delivering enough business value to justify that investment?

The least expensive agent is not necessarily the best investment. An agent that costs more but resolves substantially more cases may deserve additional capacity. An inexpensive agent that rarely completes its task may not. Governance therefore needs a second unit alongside tokens and dollars: business outcomes.

This is ultimately an ROI problem. Organizations want to understand whether their agents are creating more value than they cost. However, connecting business outcomes to the underlying cost of running an agent can be difficult.

ROI for agents in Foundry, currently in private preview, helps organizations connect agent costs to business outcomes. Teams define the outcomes they want to track, such as successful task completion, customer satisfaction, or case deflection. They then assign a business value to those outcomes and define how success should be measured. Foundry tracks which outcomes an agent achieves, and the model and tool costs incurred along the way, calculating:

• Value generated: The total value attributed to successful business outcomes.
• Total cost: The model and tool costs incurred to achieve those outcomes.
• Net value: The value remaining after costs are subtracted.
• ROI: The return generated relative to the investment required.

The dashboard shows daily trends and separates models from tool costs. Teams can compare agent versions using average value per conversation, pass rate, and improvement percentage. That makes optimization decisions defensible in business terms: not merely “the new version uses fewer tokens,” but “the new version produces more net value.”

The ROI feature also connects the business view to engineering evidence. Teams can inspect the lowest-ROI conversations and traces to find an oversized model, repetitive tool calls, or a workflow consuming tokens without producing meaningful outcomes. Because ROI is connected to observability data, teams can move directly from a business metric to the traces, evaluations, and operational signals that explain what is driving cost, quality, and business outcomes.

A low-ROI trace can point to a request that should be routed differently, context that should be removed, or an agent configuration that should be optimized. The same telemetry used to improve quality and efficiency can now help organizations answer the question the business ultimately asks: is this agent worth what it costs?

Run AI as one managed investment system

Together, the four posts in this series describe one optimization system operating at three speeds. At runtime, model routing, deployment choices, and caching right-size each request. Over days and weeks, context engineering, memory, tools, and agent optimization improve the workflow. Continuously, governance attributes consumption, enforces limits, and measures whether the portfolio is creating value.

The same evidence connects every layer, and answers different questions:

1. Traces show what an agent did on a run, exposing expensive requests and inefficient context.
2. Evaluations show whether the output was good, protecting quality as configurations change.
3. Cost attribution shows where the money went, pointing to the project, agent, or model to intervene on.
4. ROI shows whether the work was worth it, telling leaders whether to optimize an agent, give it more capacity, or retire it.

Cost is only one part of a much bigger governance story, and it helps to be clear about who owns which part.

• Foundry is built for developers creating agents. It’s where developers build, test, and optimize, and Foundry Control Plane gives them an operating view of everything they’ve shipped, from cost trends and anomalies to token usage and lifecycle controls, with Azure Policy, Microsoft Defender, and Microsoft Purview woven in so compliance and security aren’t an afterthought.
• Microsoft Agent 365 is built for the people responsible for the entire enterprise estate. IT administrators and security teams use it to discover, inventory, secure, and manage every agent in the tenant, whether it came out of Foundry, Microsoft 365, or a partner platform, and to extend the same identity, access, and data protections to agents that they already apply to people.

The FinOps capabilities we’ve covered in this series live on the Foundry side of that line, giving developers and platform teams the levers to keep spend predictable, while IT and security govern the estate around them in Agent 365.

Agent optimization isn’t about driving the cost of every request to zero. It’s about running agents with the same discipline you’d apply to any other serious investment, and that is what Foundry is built for: helping developers build and manage agents that are efficient by design, contained as they scale, and accountable for the value they create.

Get started

If you’re governing agents today, start by making their consumption visible and attributable. Identify which agents and teams are driving usage, apply request-time limits to contain unexpected consumption, and pair those controls with financial budgets and alerts. Then connect cost to business outcomes so you can decide which agents to optimize, scale, or retire.

• Enforce token limits for model deployments in Microsoft Foundry.
• Plan, monitor, and allocate AI costs in Microsoft Foundry costs.
• Create budgets and alerts in Cost Management.
• Explore AI Gateway governance capabilities in Azure API Management.
• Learn about observability and ROI for agents in Foundry.

Microsoft Foundry

The enterprise AI platform to build, ground, and govern AI apps and agents at scale.

Start managing agents

Did you miss these posts in The Economics of Agent Optimization series?

• AI cost management: From AI pilots to measurable ROI
• AI cost optimization: How to lower AI spend
• AI agent optimization: How context engineering lowers AI costs

The post The Economics of Agent Optimization: How AI agent governance controls cost and proves ROI appeared first on Microsoft Azure Blog.