Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
159398 stories
·
33 followers

Dogfooding BlocSignal on the Web: Building a 100K Ops/sec Reactive App with Jaspr and Dart 3.13

1 Share

Building Pure Dart Web Apps Without Compromise

When developers evaluate Dart for the web, they typically face a stark tradeoff:

  1. Flutter Web: Exceptional for canvas-driven applications, design systems, and cross-platform desktop/mobile parity—but heavy for content-first landing pages, docs, and fast-loading SEO sites.
  2. Jaspr Web: A lightweight, component-driven framework that compiles pure Dart to HTML and CSS with instant first paint and full search engine indexing.

When we built the official documentation and showcase site for BlocSignal, we knew Jaspr was the perfect foundation. But like many engineers diving into a new UI paradigm, our initial implementation took a shortcut: we used raw StatefulComponent lifecycles and manual .subscribe() callbacks to wire up our state machines.

It worked—but it wasn't idiomatic.

In this behind-the-scenes case study, we walk through the process of dogfooding bloc_signals_jaspr across blocsignal.dev, replacing manual subscription glue with declarative consumer components, achieving 100,000 operations/sec in compiled JavaScript, and exploring the sheer developer ergonomics of Dart 3.13 primary constructors.

The "Manual Subscription Trap": Why Raw .subscribe() Fails at Scale

In classic Flutter or Jaspr development, when you create a state machine without framework-level consumer widgets, you might be tempted to subscribe inside initState():

// ❌ THE ANTI-PATTERN: Manual subscription glue in StatefulComponent
class LiveVisualizerState extends State<LiveVisualizer> {
  late final LiveCounterBloc _bloc;

  @override
  void initState() {
    super.initState();
    _bloc = LiveCounterBloc();
    // ⚠️ Flaw 1: Every state change triggers a full component setState
    _bloc.state.subscribe((_) {
      if (mounted) setState(() {});
    });
  }

  @override
  void dispose() {
    // ⚠️ Flaw 2: Manual dispose tracking
    _bloc.close();
    super.dispose();
  }
}

While this appears harmless in a simple counter demo, it introduces three severe architectural flaws:

  1. The Double Re-Render Penalty: When a user clicks a button that calls both _bloc.add(Event()) and a local setState(), the component executes two back-to-back render passes in the exact same frame.
  2. Batch UI Thrashing: If you execute a high-frequency benchmark (e.g. dispatching 1,000 events in a loop), manual subscriptions attempt to invoke setState() 1,000 times during the loop, creating massive JS event-loop thrashing.
  3. Coarse-Grained Rebuilds: The entire component tree rebuilds on every change, even if only a single badge or numeric label changed value.

To solve this, we brought the full power of bloc_signals_flutter's declarative consumer components over to Jaspr in bloc_signals_jaspr.

1. Declarative Routing with a NavigationCubit

Rather than relying on ad-hoc URL parsing scattered across components, we modeled the entire site navigation as a pure Dart state machine:

// lib/src/cubits/navigation_cubit.dart
import 'dart:js_interop';
import 'package:bloc_signals_jaspr/bloc_signals_jaspr.dart';
import 'package:web/web.dart' as web;

@JS('trackGaPageView')
external void _trackGaPageView(JSString path);

class NavigationCubit() extends CubitSignal<String> {
  this : super(initialState: _resolveCurrentPath()) {
    // Listen to browser history navigation
    web.window.addEventListener('popstate', ((web.Event _) => _sync()).toJS);
    web.window.addEventListener('hashchange', ((web.Event _) => _sync()).toJS);
    _trackPageView(stateValue);
  }

  static String _resolveCurrentPath() {
    final path = web.window.location.pathname;
    final hash = web.window.location.hash.toLowerCase();
    if (path.startsWith('/showcase') || hash.contains('showcase')) return '/showcase';
    if (path.startsWith('/minesweeper') || hash.contains('minesweeper')) return '/minesweeper';
    if (path.startsWith('/publications') || hash.contains('publications')) return '/publications';
    return '/';
  }

  void _sync() {
    final next = _resolveCurrentPath();
    if (next != stateValue) {
      emit(next);
      _trackPageView(next);
    }
  }

  void _trackPageView(String route) {
    try {
      _trackGaPageView(route.toJS);
    } catch (_) {}
  }
}

At the root of the application, we inject the cubit using BlocSignalProvider and build the active page using BlocSignalBuilder:

// lib/src/app.dart
class const App({super.key}) extends StatelessComponent {
  @override
  Component build(BuildContext context) {
    return BlocSignalProvider<NavigationCubit>(
      create: (_) => NavigationCubit(),
      child: const _AppRouter(),
    );
  }
}

class const _AppRouter() extends StatelessComponent {
  @override
  Component build(BuildContext context) {
    return BlocSignalBuilder<NavigationCubit, String>(
      builder: (context, currentPath) => switch (currentPath) {
        '/showcase' => const ShowcasePage(),
        '/minesweeper' => const MinesweeperPage(),
        '/publications' => const PublicationsPage(),
        _ => const HomePage(),
      },
    );
  }
}

Now, anywhere in the component tree—such as our sticky navigation header—we can reactively highlight active links with zero prop-drilling using context.select():

// lib/src/components/navbar.dart
final activePath = context.select<NavigationCubit, String>((c) => c.stateValue);

a(
  href: '/showcase',
  classes: activePath == '/showcase' ? 'nav-active' : '',
  [Component.text('Showcase')],
)

2. Fine-Grained DOM Updates with BlocSignalSelector

On the blocsignal.dev homepage, the Interactive Live Visualizer demonstrates real-time state updates across multiple metrics:

  1. Primary State: The raw integer count.
  2. Computed State (2x): state * 2.
  3. Parity & Status: EVEN / ODD and POSITIVE / NEGATIVE / ZERO.

Instead of rebuilding the entire visualizer card on every tick, each card uses BlocSignalSelector to isolate its DOM mutations:

// 1. Primary Count Selector
BlocSignalSelector<LiveCounterBloc, int, int>(
  selector: (state) => state,
  builder: (context, count) => span(classes: 'metric-value', [
    Component.text('$count'),
  ]),
),

// 2. Computed 2x Doubled Selector
BlocSignalSelector<LiveCounterBloc, int, int>(
  selector: (state) => state * 2,
  builder: (context, doubled) => span(classes: 'metric-value', [
    Component.text('$doubled'),
  ]),
),

// 3. Record-based Multi-Value Selector
BlocSignalSelector<LiveCounterBloc, int, ({String parity, String status})>(
  selector: (state) => (
    parity: state % 2 == 0 ? 'EVEN' : 'ODD',
    status: state > 0 ? 'POSITIVE' : (state < 0 ? 'NEGATIVE' : 'ZERO'),
  ),
  builder: (context, derived) => div(classes: 'status-row', [
    span(classes: 'chip', [Component.text(derived.parity)]),
    span(classes: 'chip', [Component.text(derived.status)]),
  ]),
)

Buttons dispatch events directly using context.read<LiveCounterBloc>():

button(
  classes: 'btn-increment',
  onClick: () => context.read<LiveCounterBloc>().add(IncrementEvent()),
  [Component.text('+ 1 Increment')],
)

And background telemetry logging is captured cleanly with BlocSignalListener:

BlocSignalListener<LiveCounterBloc, int>(
  listener: (context, state) {
    _appendLog('⚡ TRANSITION -> State: $state [0ms Synchronous]');
  },
  child: visualizerMarkup,
)

3. Side-by-Side: Traditional Dart 3.5 vs. Modern Dart 3.13

Because our website is an application rather than a published library package, we can take full advantage of Dart 3.13 primary constructors and constructor shorthands.

Look at the difference in boilerplate when defining a reactive Jaspr card:

Traditional Dart 3.5 Syntax

class MetricCard extends StatelessComponent {
  const MetricCard({
    required this.title,
    required this.value,
    super.key,
  });

  final String title;
  final String value;

  @override
  Component build(BuildContext context) {
    return div(classes: 'metric-card', [
      span([Component.text(title)]),
      h3([Component.text(value)]),
    ]);
  }
}

Modern Dart 3.13 Syntax

class const MetricCard(final String title, final String value, {super.key}) 
    extends StatelessComponent {
  @override
  Component build(BuildContext context) {
    return div(classes: 'metric-card', [
      span([Component.text(title)]),
      h3([Component.text(value)]),
    ]);
  }
}

By placing fields directly in the primary constructor parameter list, 5 lines of boilerplate collapse into a clean, single-line class header with zero repetition.

4. Web Performance Reality: 100K Operations/Sec in Browser JavaScript

One of the biggest surprises for developers testing the live visualizer on blocsignal.dev is the built-in stress test:

Benchmark: Dispatches 1,000 synchronous transitions in a tight loop.

In traditional stream-based architectures (like classic BLoC or Rx on the web), dispatching 1,000 events allocates 1,000 StreamController events and queues 1,000 microtask hops through Dart's async runtime.

In BlocSignal, state changes propagate through a synchronous dependency graph:

  • 0 Microtask Queue Hops: State transitions resolve in the exact same call stack.
  • 0 Intermediate Frame Tearing: The DOM settles cleanly without intermediate stutter.
  • Throughput: Even running in compiled JS inside a standard browser tab, it clocks over 100,000 operations per second (~10ms for 1,000 full event-state cycles).

Summary & Live Demo

Dogfooding bloc_signals_jaspr on our own production website proved that building web applications in pure Dart doesn't require choosing between developer discipline and raw performance:

Feature Classic Web BLoC / Rx bloc_signals_jaspr
Reactivity Latency Microtask Queue Delay (Async) 0ms Synchronous Call Stack
Component Wiring Manual subscribe / dispose Declarative BlocSignalBuilder
DOM Rebuild Scoping Coarse Component Rebuilds Fine-Grained BlocSignalSelector
JS Web Throughput ~2,000 – 10,000 ops/sec ~100,000+ ops/sec
Syntax Overhead Verbose Field & Constructor Maps Dart 3.13 Primary Constructors

You can try the interactive visualizer and play the live Minesweeper case study right now at blocsignal.dev!

All the source code is open source and visible directly in our GitHub monorepo at RandalSchwartz/BlocSignal. ⭐️

Read the whole story
alvinashcraft
21 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Serving Gemma4 with Rust on vLLM 🦀

1 Share

This tutorial walks through installing and setting up the Rust toolchain for vLLM on an
AWS EC2 G5g instance — Graviton2 (aarch64) with an NVIDIA T4G GPU — and getting vLLM's
Rust frontend (vllm-rs) built, running, and verified.

This paper is a follow-on to the original G5g Gemma 4 build.

Everything below was run on the box. 🦀

Wait, vLLM has Rust in it?

You betcha. Since PR #40848 (merged
2026-05-21), vLLM vendors a 14-crate Rust workspace:

bench  chat  cmd  engine-core-client  llm  managed-engine  metrics
mock-engine  parser  parser/python  server  text  tokenizer  tracing

Edition 2024, resolver 3. Straight from the vendored rust/Cargo.toml:

Crate Version Job
axum 0.8.8 the HTTP server
tokio 1.47.1 async runtime
zeromq 0.6.0 talks to the Python engine
rmp-serde / rmpv 1.3.1 msgpack on the wire
minijinja 2.22 chat templates
tonic / prost 0.14.6 / 0.14.3 gRPC — remember this one

It's a drop-in replacement for the Python FastAPI server. Two artifacts get built:

  • 🦀 vllm-rs — the axum frontend binary
  • 🐍 vllm._rust_tool_parser — a PyO3 extension module

Rust is a build requirement now

That's the headline, and it's reason enough on its own: you cannot build vLLM from source at
v0.27.2rc0 without Rust in the picture.
setup.py imports it at module scope, line 21,
unguarded:

from setuptools_rust.build import build_rust

No try, no feature flag, no opt-out. Metadata generation doesn't happen without it.

And this isn't a quirk of one release. vLLM's Rust surface is 14 crates covering the HTTP
frontend, the tool parser, the tokenizer and the benchmark client, and it has been growing
since it landed. If you build inference infrastructure from source, a Rust toolchain is
becoming table stakes — so it's worth knowing how to drive it properly rather than working
around it.

Three things do get conflated, though, and they have different scopes:

Component Needed to build vLLM? Needed to serve?
setuptools_rust (Python pkg) yes, always no
cargo / rustc toolchain for working Rust artifacts no
protoc for vllm-rs specifically no

Then why doesn't pip install vllm need this?

Because normally pip installs it for you. pyproject.toml declares it:

[build-system]
requires = [
    "cmake>=3.26.1", "ninja", "packaging>=24.2",
    "setuptools>=77.0.3,<81.0.0", "setuptools-scm>=8.0",
    "setuptools-rust>=1.9.0",          # <- pip grabs this automatically
    "torch == 2.13.0",                 # <- ...and this. Which is the problem.
    "wheel", "jinja2",
]

Under normal build isolation, pip creates a clean env, installs that list, and builds.
You never see setuptools_rust because you never had to think about it.

But look at the torch pin. Building in isolation means pip installs torch 2.13.0 from
PyPI
— and the PyPI aarch64 wheels are built for sm_80 and up. No sm_75. Which
destroys the entire reason for building from source on a T4G.

So on this box you must build against the DLAMI's own torch, and that means:

python use_existing_torch.py
pip install -e . --no-build-isolation

--no-build-isolation turns off the automatic install of everything in that requires
list.
From that moment on, every build dependency is yours to supply by hand — including
setuptools_rust, which is why it turns up as a bare ModuleNotFoundError minutes into a
build that has nothing visibly to do with Rust.

So the toolchain was always required; isolation was just hiding it. Building this way means
you own the dependency list, which is the rest of this walk-through. ⚡

What the DLAMI gives you, and what it doesn't

The AWS Deep Learning ARM64 AMI ships a runtime, not a build environment. On a fresh box:

Thing Present?
PyTorch 2.12 with sm_75
NVIDIA driver
nvcc / CUDA toolkit
Rust toolchain
setuptools_rust
protoc

Four of those six are on you. Let's install them.

Step 1 — Rust itself

Standard rustup, nothing aarch64-specific about it:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
. "$HOME/.cargo/env"
stable-aarch64-unknown-linux-gnu installed - rustc 1.97.1 (8bab26f4f 2026-07-14)

Rust is installed now. Great!

Note the triple: stable-aarch64-unknown-linux-gnu. Rust's aarch64 support is a complete
non-event
, which is a lovely change of pace on this hardware. ⚡

Step 2 — setuptools-rust

python3 -m pip install setuptools_rust

Per the section above: --no-build-isolation means pip won't do this for you. Do it early
— the failure lands during metadata generation, minutes into a build, as a bare
ModuleNotFoundError: No module named 'setuptools_rust' nowhere near anything that looks
like Rust.

⚠️ Install it into the same interpreter you'll build with. On the DLAMI that's
/opt/pytorch/bin/python3, not the system python3 — they're different, and the one that
matters is whichever owns the torch you're building against.

Step 3 — protoc 🔎

This is the one nobody documents:

apt-get install -y protobuf-compiler
protoc --version
libprotoc 3.21.12

Why: vllm-rs depends on the vllm-server crate, vllm-server builds gRPC stubs with
tonic/prost, and prost-build shells out to protoc. Skip it and the frontend binary
does not get built — see the summary at the end for how loudly that doesn't fail.

The tool parser has no protobuf dependency, which is why it builds either way.

Step 4 — the CUDA toolkit, while you're here

Not Rust, but the same class of problem, and you need it for vLLM's kernels:

# NVIDIA's **sbsa** repo — not the x86 one, easy reflex to get wrong on Arm
apt-get install -y cuda-toolkit-13-2

Step 5 — build the Rust artifacts

cd /opt/vllm-src
python tools/build_rust.py --release

⚠️ Do not omit --release. setuptools-rust builds inplace targets in debug by default,
and pip install -e . is an inplace build. The difference is not subtle:

Artifact Debug Release
_rust_tool_parser.abi3.so 100,913,216 B 1,009,080 B

100x. The debug artifact is four times the size of every CUDA kernel in vLLM combined.

Timing on a g5g.xlarge (4 vCPU), cold:

real    9m1.746s
user    25m9.199s
sys     1m35.023s

501 crates. Zero warnings. Exit 0. 🟢

Rust's aarch64 support does not put up a fight here — which is a pleasant contrast with the
CUDA side of this box, where SM 7.5 on Graviton needs a custom arch list and a patched
kernel.

Step 6 — check what you got

ls -la vllm/vllm-rs vllm/_rust_tool_parser.abi3.so
-rwxr-xr-x 1 root root 50039024 vllm/vllm-rs
-rwxr-xr-x 1 root root  1009080 vllm/_rust_tool_parser.abi3.so
file vllm/vllm-rs
ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV),
dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, not stripped
vllm/vllm-rs --help
Rust frontend and managed-engine CLI for vLLM.

Commands:
  frontend  Run the Rust OpenAI frontend as a Python-supervised worker
  serve     Launch a managed Python headless engine, then run the Rust OpenAI frontend
  bench     Run vLLM benchmarks
  render    Run engine-free request rendering and preprocessing

If vllm/vllm-rs isn't there, go back to Step 3.

Step 7 — run it, and mind the entrypoint ⚠️

VLLM_USE_RUST_FRONTEND=1 vllm serve google/gemma-4-E2B-it \
  --dtype float16 \
  --kv-cache-dtype auto \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.90 \
  --max-num-seqs 8 \
  --tensor-parallel-size 1 \
  --host 0.0.0.0 --port 8000

It must be vllm serve. If you launch the module directly —

# ❌ VLLM_USE_RUST_FRONTEND is IGNORED here
python -m vllm.entrypoints.openai.api_server --model--host 0.0.0.0 --port 8000

— the variable does nothing. No warning, no Unknown vLLM environment variable line. The
server comes up healthy and serves happily on the Python frontend, and a benchmark run
against it looks entirely normal.

The flag is read in exactly two places:

vllm/entrypoints/cli/serve.py:62        envs.VLLM_RUST_FRONTEND_PATH if envs.VLLM_USE_RUST_FRONTEND else None
vllm/entrypoints/openai/dp_supervisor.py:261   if envs.VLLM_USE_RUST_FRONTEND and envs.VLLM_RUST_FRONTEND_PATH:

api_server.py never mentions it.

How do I know it's actually Rust? 🔎

Three checks. Do all three the first time.

1. The server: header:

curl -si localhost:8000/health | grep -i '^server:'
Frontend Response
🐍 Python server: uvicorn
🦀 Rust (no server: header at all)

2. The process:

pgrep -af vllm-rs
26588 /opt/vllm-src/vllm/vllm-rs frontend --listen-fd 17
  --input-address  ipc:///tmp/f60f3962-d45b-4bcd-9026-c0dc32736028
  --output-address ipc:///tmp/5f75411d-2787-43bb-b4fc-14bf504a1cce
  --engine-start-index 0 --engine-count 1 --data-parallel-size 1

3. The log prefix(RustFrontend pid=…) instead of (APIServer pid=…):

INFO [utils.py:392] Launching Rust frontend: /opt/vllm-src/vllm/vllm-rs frontend --listen-fd 17 …

So where does Rust actually sit?

In two places, and they're quite different. One is a separate process; the other is a
shared object loaded inside the Python process. Here's the whole VM:

┌─ EC2 g5g.4xlarge ── Graviton2, aarch64 ─────────────────────────────────────┐
│                                                                             │
│  Deep Learning ARM64 AMI · Ubuntu 24.04 · NVIDIA driver 595.71.05           │
│  you add > cuda-toolkit-13-2 (sbsa) · rustup 1.97.1 · protobuf-compiler     │
│                                                                             │
│      HTTP :8000                                                             │
│          |                                                                  │
│          v                                                                  │
│  ┌───────────────────────────────┐                                          │
│  │ [RUST] vllm-rs                │  50 MB aarch64 ELF, its OWN process      │
│  │        axum 0.8.8 · tokio     │  built from the vendored rust/ workspace │
│  │        minijinja · fastokens  │  <- Step 5                               │
│  └────────┬─────────────▲────────┘                                          │
│           |             |                                                   │
│  ipc://   | ROUTER      | PULL     msgpack (rmp-serde / rmpv)               │
│           v             |                                                   │
│  ┌────────┴─────────────┴────────┐                                          │
│  │ [PY]   vLLM supervisor        │  `vllm serve` opens the socket, then     │
│  │                               │  hands listen-fd 17 down to vllm-rs      │
│  └────────┬──────────────────────┘                                          │
│           | spawns                                                          │
│           v                                                                 │
│  ┌───────────────────────────────┐                                          │
│  │ [PY]   EngineCore             │  torch 2.12.0+cu132, arch list has sm_75 │
│  │  ┌─────────────────────────┐  │                                          │
│  │  │ [RUST] _rust_tool_parser│  │  PyO3 .so LOADED INTO the Python         │
│  │  │        1.0 MB release   │  │  process — not a process of its own      │
│  │  └─────────────────────────┘  │                                          │
│  └────────┬──────────────────────┘                                          │
│           | CUDA                                                            │
│           v                                                                 │
│  ┌───────────────────────────────┐                                          │
│  │ NVIDIA T4G · SM 7.5           │  15,360 MiB GDDR6 · 277 GB/s measured    │
│  │ TRITON_ATTN kernels           │  weights 9.94 GiB · KV 2.95 GiB          │
│  └───────────────────────────────┘                                          │
└─────────────────────────────────────────────────────────────────────────────┘

Two things worth pulling out of that picture:

  • vllm-rs is not a sidecar you point at a port. The Python side opens the listening socket and passes the file descriptor down. It's a worker the supervisor forks and feeds.
  • _rust_tool_parser is Rust living inside Python. It's the one that always builds (no protoc needed), which is why a broken install still leaves Rust on the box — just not the Rust you wanted.

And note where the GPU sits relative to all of this: at the bottom, behind everything. That's
the reason the benchmark below comes out the way it does.

Bonus: there's a Rust benchmark client too

VLLM_USE_RUST_BENCH=1 vllm bench serve …

Same binary, bench subcommand. Requires VLLM_RUST_FRONTEND_PATH to resolve, so it needs
the same Step 3 → Step 5 you just did.

Two warnings you should not scroll past 🔴

The server came up healthy. These went by in the startup log anyway.

Gemma 4 defeats the fast tokenizer:

INFO    [hf.rs:200] loading tokenizer with fastokens
WARNING [hf.rs:221] failed to load tokenizer with fastokens; falling back to
        HuggingFace tokenizers
        error=tokenizer error: normalizer error: unsupported normalizer type: Replace

fastokens 0.2.1 doesn't implement the Replace normalizer that Gemma 4's tokenizer.json
uses, so it falls back to the same HuggingFace tokenizers the Python path uses. Note the
fallback is graceful and correct — you just don't get the fast path on this model yet. It's
a coverage gap in a young crate, and one normalizer away from closing.

Multimodal isn't wired up for this model:

WARNING [multimodal.rs:446] multimodal model spec is not registered; disabling
        image/video support   model_id="google/gemma-4-E2B-it" model_type="gemma4"

Gemma 4 E2B is a vision model, and gemma4 isn't in the Rust multimodal spec table yet.
Text requests behave identically and the endpoint is healthy, so nothing in a normal check
reveals it. Also a registration gap rather than a design problem — but check it for your model
before you switch, because a healthy endpoint won't tell you.

Is it faster?

On a T4G, no. Output token throughput, same engine config, client on the box against
localhost:

Concurrency 🐍 Python 🦀 Rust
1 28.65 29.30
4 97.48 97.26
8 168.33 169.39
16 169.96 170.19
32 170.99 170.34

Median TTFT tracks just as tightly — 14305 ms against 14311 ms at concurrency 32.

That's the expected result, and worth saying plainly: decode on this card is
bandwidth-bound at a measured 277 GB/s, and the engine saturates at --max-num-seqs 8.
A frontend rewrite targets CPU-side per-request overhead. Here that overhead hides behind the
GPU, so swapping it can't move a bottleneck-limited number. If you want the Rust frontend
to buy you tokens per second on a small GPU, it won't.

One signal does appear, in median inter-token latency at high concurrency:

Concurrency 🐍 Python 🦀 Rust Δ
16 38.55 36.18 −6.4%
32 38.41 36.23 −5.9%

Mean TPOT barely moves, so this is the middle of the distribution tightening rather than
everything speeding up — the shape you'd expect from a frontend scheduling streaming work
more evenly once many streams are in flight. Worth knowing if you serve at concurrency; not
worth switching for on its own. 📊

If a plain pip install -e . already ran

A from-source vLLM install done without the steps above succeeds, exits 0, and leaves you
with a 96 MB debug tool parser and no frontend binary. Four defaults stack up to make that
silent:

Symptom Cause Fix
No vllm/vllm-rs after a clean build protoc absent ⇒ vllm-server fails with code 101 Step 3
pip install exits 0 anyway optional=not should_require_rust_frontend() — setuptools-rust swallows it VLLM_REQUIRE_RUST_FRONTEND=1
_rust_tool_parser.abi3.so is ~96 MB editable ⇒ inplace ⇒ debug profile --release
FileNotFoundError: … vllm-rs was not found the above, discovered at import time Steps 3 + 5
Healthy server, but server: uvicorn flag set on the api_server module, which never reads it vllm serve

VLLM_REQUIRE_RUST_FRONTEND=1 turns the second row into a hard build failure, which is what
you want on any machine you plan to serve from.

Cheat sheet

# toolchain — you supply these by hand because the sm_75 requirement
# forces --no-build-isolation, which disables pip's automatic build deps
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
. "$HOME/.cargo/env"
/opt/pytorch/bin/python3 -m pip install setuptools_rust   # the BUILD interpreter
apt-get install -y protobuf-compiler cuda-toolkit-13-2

# build against the DLAMI's torch, not a PyPI one (PyPI aarch64 has no sm_75)
cd /path/to/vllm
python use_existing_torch.py
TORCH_CUDA_ARCH_LIST=7.5 VLLM_REQUIRE_RUST_FRONTEND=1 \
  pip install -e . --no-build-isolation

# Rust artifacts, release profile (editable installs default to debug: 100x bigger)
VLLM_REQUIRE_RUST_FRONTEND=1 python tools/build_rust.py --release

# confirm
ls -la vllm/vllm-rs && vllm/vllm-rs --help

# run — `vllm serve`, NOT the api_server module
VLLM_USE_RUST_FRONTEND=1 vllm serve <model> --host 0.0.0.0 --port 8000

# verify it's really Rust
curl -si localhost:8000/health | grep -i '^server:'   # Rust sends none
pgrep -af vllm-rs

Run on EC2 g5g.xlarge and g5g.4xlarge, us-east-1a, NVIDIA T4G (SM 7.5). vLLM
0.27.2rc1.dev0+g7f7a32cfe, rustc 1.97.1, setuptools-rust 1.13.0, libprotoc 3.21.12,
torch 2.12.0+cu132. Benchmarks are one run per cell for Rust and two for Python; treat the
TPOT delta as suggestive.

Read the whole story
alvinashcraft
21 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

RNR 370 - CopilotKit with Mike Ryan

1 Share

Robin and Mazen talk with Mike Ryan of CopilotKit about bringing AI agents to React Native apps. They break down AG-UI, generative UI, shared state, and the guardrails developers need to build useful, trustworthy mobile experiences.

 

Show Notes

  1. CopilotKit: Bring Users and AI Agents together inside real apps

 

Connect With Us!

 

Sponsored by Infinite Red

Infinite Red is a premier mobile app consultancy, especially focused on Expo and React Native, located fully remote in the US. We’re a team of 30 with highly experienced mobile app developers and have been doing this for over a decade. We are also one of the first development teams to adopt agentic coding in a way that keeps high quality standards and aren’t afraid to do things the old school way if we need to. If you’re looking for mobile app or React Native or Expo expertise for your next project, hit us up at infinite.red/radio.





Download audio: https://cdn.simplecast.com/media/audio/transcoded/1208ee61-9c16-43c1-bc4c-ca790717f4a8/2de31959-5831-476e-8c89-02a2a32885ef/episodes/audio/group/220a20e4-2625-473e-8b16-6e06389afbe6/group-item/5abcc824-fe72-452b-8ccd-605efc8f7832/128_default_tc.mp3?aid=rss_feed&feed=hEI_f9Dx
Read the whole story
alvinashcraft
22 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

PPP 519 | Overcoming the Inner Propaganda That Sabotages Projects and Teams, with Owen Fitzpatrick

1 Share

Summary

In this episode, Andy sits down with Owen Fitzpatrick, a psychologist, speaker, and author of Inner Propaganda: Leading Hearts and Minds through Turbulent Times. Owen has spent close to 30 years studying how beliefs form and change, interviewing people everywhere from North Korea to Rwanda to Afghanistan. His thesis is unsettling: our brains do not simply take in facts and reach objective conclusions. They build a story we then experience as reality.

Owen and Andy work through what that means on real projects. You'll hear how a warning from a colleague can quietly harden into a conviction about a teammate, and how Bayesian reasoning gives you a way out. You'll learn Owen's five types of truth, how to tell courageous conviction from dangerous denial, and what leaders can actually make stable when they cannot promise a stable outcome. Owen also explains why pushing harder for buy-in is often the very reason people resist, and how an antifragile identity helps teams face uncertainty like AI without denial or panic.

If you're looking for a fresh way to think about belief, influence, and leading through turbulent times, this episode is for you!

Sound Bites

  • "We all live in that world where we think we're the one person that isn't the victim of propaganda, and my point or my thesis is we're all victims of our own inner propaganda."
  • "We're not necessarily just convinced by others. We convince ourselves."
  • "Because I think when we say, 'I'm no good at something,' we lock ourselves into it."
  • "Well, if you're not great at communicating with people, get great."
  • "Most of the time our beliefs just create a sort of a reality for us, and that reality can help us or harm us."
  • "So our brains are prediction machines."
  • "Whenever we talk about belief, believing in your ability to succeed in the future is critical if you want to succeed in the future, but that doesn't mean you deny the present."
  • "It, it's not that we think negatively, it's we believe negatively, and we see the world through those lenses."
  • "So I think we want to be able to challenge our beliefs and build a bit more and get more comfortable with uncertainty away from the table, but when we're at the table with our team, that's when we bring the certainty."
  • "We like the idea that we are making this decision of our own free will."
  • "And when you look at what a belief is, a belief is an idea we feel certain about, and that word feel is the most important word of that sentence."
  • "It's okay to believe less in certain things. It's okay to believe more in certain things. It's okay to believe better."

Chapters

  • 00:00 Introduction
  • 02:29 Start of Interview
  • 02:40 The Belief About Himself Owen Took Too Long to Update
  • 05:56 A Sweaty Debate Speech and What Came After
  • 08:28 Not Good at Something Is a Skill Gap, Not an Identity
  • 09:45 The Belief Growth Mindset
  • 12:55 The Sandra Story: When a Warning Becomes a Conviction
  • 14:07 Why the Brain Craves Cognitive Closure
  • 17:15 Using Bayesian Reasoning to Loosen a Belief
  • 21:00 Melanie Perkins and Alan Mulally: Conviction or Denial?
  • 21:36 The Five Types of Truth
  • 26:24 Just Because It Is Your Truth Does Not Make It True
  • 28:41 Where Objective Truth Actually Belongs
  • 31:47 What "Leadership Is Propaganda" Does Not Mean
  • 34:45 Why Change Management and AI Adoption Stall
  • 36:30 Why Owen Chose Propaganda Over Self-Talk
  • 37:57 Do I Believe It Because It Makes Me Look Good?
  • 38:49 Answering the AI Question Without Denial or Hype
  • 40:15 Building an Antifragile Identity
  • 44:27 Certainty Is Contagious, and When to Change Course
  • 45:20 What Leaders Can Make Stable in a Volatile Period
  • 47:45 Control What You Can, Influence What You Can, Accept the Rest
  • 51:01 When Pushing for Buy-In Creates the Resistance
  • 52:45 You're Asking the Wrong Question About Persuasion
  • 54:30 Reading the Person Before You Make the Ask
  • 57:30 Helping Kids Hold Strong Beliefs Without Contempt
  • 1:01:46 End of Interview
  • 1:02:15 Andy Comments After the Interview
  • 1:05:09 Outtakes

Learn More

You can learn more about Owen and his work at InnerPropaganda.com.

For more learning on this topic, check out:

  • Episode 370 with Chantel Prat. One of the smartest, clearest, and funniest books on the brain, and why all of it matters for how you lead.
  • Episodes 59 and 60 with Cathy Davidson. She explains how the brain science of attention changes everything.
  • Episode 32 with Brad Kolar. A look at the direct implications of neuroscience for leadership.
  • Owen's TED Talk, available here on YouTube.

Chat with PMeLa

You can chat directly with PMeLa, the podcast's AI persona, to get episode recommendations and answers to your project management and leadership questions. Visit PeopleAndProjectsPodcast.com/PMeLa to chat with her.

Join Us for LEAD52

I know you want to be a more confident leader–that's why you listen to this podcast. LEAD52 is a global community of people like you who are committed to transforming their ability to lead and deliver. It's 52 weeks of leadership learning, delivered right to your inbox, taking less than 5 minutes a week. And it's all for free. Learn more and sign up at GetLEAD52.com. Thanks!

Thank you for joining me for this episode of The People and Projects Podcast!

Talent Triangle: Power Skills

Topics: Leadership, Belief, Persuasion, Influence, Uncertainty, Change Management, Growth Mindset, Psychological Reactance, Buy-In, Artificial Intelligence, Resilience, Project Management

The following music was used for this episode:

Music: The Fantastical Ferret by Tim Kulig
License (CC BY 4.0): https://filmmusic.io/standard-license

Music: Funny by Frank Schroeter
License (CC BY 4.0): https://filmmusic.io/standard-license





Download audio: https://traffic.libsyn.com/secure/peopleandprojectspodcast/519-OwenFitzpatrick.mp3?dest-id=107017
Read the whole story
alvinashcraft
22 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Best of Big Technology: How Ozempic Changes Our Bodies, Minds, and Economy — With Johann Hari

1 Share

Johan Hari is a New York Times bestselling author and has a new book, Magic Pill, about his experience with weight loss drugs. Hari joins Big Technology to discuss the impact of Ozempic on our relationship with food and the economy. Tune in to hear how these medications work, their benefits, and their potential risks. We also cover the science behind these drugs, economic implications, and ethical considerations. Hit play for an in-depth conversation on the future of weight loss drugs and their broader societal effects


---

Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.

Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b

Stop online threats before they become real-world attacks. Visit ⁠ironwall.com/BIGTECHNOLOGY⁠ and request a free Risk Assessment to see exactly how exposed your executives are

Learn more about your ad choices. Visit megaphone.fm/adchoices





Download audio: https://pdst.fm/e/tracking.swap.fm/track/t7yC0rGPUqahTF4et8YD/pscrb.fm/rss/p/traffic.megaphone.fm/AMPP4113612310.mp3
Read the whole story
alvinashcraft
22 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Kimi K3 in GitHub Copilot, Multi-Agent AI, and more | The Upload

1 Share
From: Microsoft Developer
Duration: 2:15
Views: 704

Kimi K3 just landed in GitHub Copilot, Microsoft published new guidance for building multi-agent systems, and Microsoft Foundry leaned further into a multi-model strategy. Here's your weekly roundup of the Microsoft developer news worth paying attention to.

Episode 2 of The Upload, a weekly roundup of Microsoft developer news worth paying attention to. Hosted by Cameron Tomisser.

In this episode:
- Kimi K3 is now available in GitHub Copilot — another frontier-class model option across Visual Studio, VS Code, GitHub, and the Copilot CLI
- New Microsoft guidance on building multi-agent systems with Microsoft Agent Framework
- Microsoft Foundry's expanding model ecosystem — model portfolios, evaluation, optimization, and governance over a single flagship model

Resources:
-Kimi K3 is now available in GitHub Copilot: https://github.blog/changelog/2026-08-06-kimi-k3-is-now-available-in-github-copilot/
-Supported AI models in GitHub Copilot: https://docs.github.com/copilot/reference/ai-models/supported-models
-Subscribe to Microsoft Developer for more practical developer and AI learning:
https://www.youtube.com/@MicrosoftDeveloper

#GitHubCopilot #KimiK3 #MicrosoftFoundry #AIAgents #MicrosoftAgentFramework

Read the whole story
alvinashcraft
22 minutes ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories