Elizabeth Stone is the Chief Product and Technology Officer (CPTO) at Netflix, where she oversees Engineering, Product, and Design. Since her first appearance on the podcast two years ago—which remained my second-most-popular episode for more than a year—she has expanded her role to lead product, in addition to engineering. Before Netflix, Elizabeth was VP of Science at Lyft, Chief Operating Officer at Nuna, an economist at Analysis Group, and a trader at Merrill Lynch.

In our in-depth conversation, we discuss:

1. Why “systems thinking” is now the most important skill she looks for

2. How to manage the flood of AI-generated output without losing quality or signal

3. How Netflix thinks about AI fluency as a universal expectation rather than a level-specific skill

4. What “excellence as an operating system” means

—

Brought to you by:

WorkOS—Make your app enterprise-ready, with SSO, SCIM, RBAC, and more: https://workos.com/lenny

Mercury—Radically different banking, now with Command: https://mercury.com/command?utm_source=lennys&utm_medium=sponsored_newsletter&utm_campaign=26q3_brand_campaign

—

Episode transcript: https://www.lennysnewsletter.com/p/netflix-cpto-on-ai-and-the-future

—

Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0

—

Where to find Elizabeth Stone:

• LinkedIn: https://www.linkedin.com/in/elizabeth-stone-608a754

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction

(02:25) AI and role confusion: the storming phase before the forming phase

(07:36) How roles have changed in the past two and a half years

(11:55) Will functions survive? The case for craft specialism

(13:26) What Netflix is hiring more of—and less of

(17:22) Why systems thinking is the rising skill across every function

(20:20) Is the design process dead?

(22:08) Skills trending down

(28:33) AI fluency and Netflix’s career ladder overlay

(31:00) AI use cases beyond coding

(35:12) Netflix’s AI history

(38:36) Excellence as an operating system

(41:11) The pillars of the excellence OS

(46:41) The keeper’s test—and why it’s mostly a positive conversation

(50:21) Attracting top talent in the age of frontier AI labs

(52:54) Junior talent, craft mastery, and the mentorship question

(56:25) Where engineering goes in 5 to 10 years

(59:45) The future of entertainment: beyond film and TV

(1:02:18) AI in Hollywood: Netflix’s creator-enablement position

(1:06:15) Lightning round and final thoughts

—

Referenced:

• How Netflix builds a culture of excellence | Elizabeth Stone (CTO): https://www.lennysnewsletter.com/p/how-netflix-builds-a-culture-of-excellence

• Brian Chesky’s new playbook: https://www.lennysnewsletter.com/p/brian-cheskys-contrarian-approach

• The design process is dead. Here’s what’s replacing it. | Jenny Wen (head of design at Claude): https://www.lennysnewsletter.com/p/the-design-process-is-dead

• Claude Code: https://www.anthropic.com/product/claude-code

• Claude Cowork: https://www.anthropic.com/product/claude-cowork

• Netflix’s “Keeper Test” and Why You Need It | Lorne Rubis: https://www.highlights.lornerubis.com/2015/08/the-netflix-keeper-test-and-the-courage-to-take-it

• Innovation for Filmmaking, By Filmmakers: Why InterPositive Is Joining Netflix: https://about.netflix.com/en/news/why-interpositive-is-joining-netflix

• InterPositive: https://weareinterpositive.com

• Netflix Prize: https://en.wikipedia.org/wiki/Netflix_Prize

• Quarterback on Netflix: https://www.netflix.com/title/81482895

• The Bill Simmons Podcast on Netflix: https://www.netflix.com/title/82186214

• Spencer Pratt on Instagram: https://www.instagram.com/spencerpratt

• Salman Rushdie’s Substack: https://salmanrushdie.substack.com

• Remarkably Bright Creatures on Netflix: https://www.netflix.com/title/81911351

• Eight Sleep: https://www.eightsleep.com

• Tour de France: https://www.letour.fr/en

—

Recommended books:

• Thinking in Systems: https://www.amazon.com/Thinking-Systems-Donella-H-Meadows/dp/1603580557

• Into Thin Air: A Personal Account of the Mt. Everest Disaster: https://www.amazon.com/Into-Thin-Air-Personal-Disaster/dp/0385494785

• Liar’s Poker: https://www.amazon.com/Liars-Poker-Norton-Paperback-Michael/dp/039333869X

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

To hear more, visit www.lennysnewsletter.com



Download audio: https://api.substack.com/feed/podcast/205675851/497b10441135a842dbe35a6891de72bb.mp3

Articles & Tutorials

Sponsored

Advertise to more than 80k Android developers! We reach out to more than 80k Android developers around the world, every week, through our email newsletter and social media channels. Advertise your Android development related service or product!

Change the Map: Feature Flags and Remote Config in Kotlin Multiplatform KMP Bits walks through feature flag and remote config patterns for safely rolling out features in Kotlin Multiplatform.

Test & Set Jesse Wilson explains how testAndSet() simplifies concurrent code by separating business logic from concurrency logic.

Our Droidcon US 2026 Picks Sam Hill highlights must-watch talks from Droidcon US 2026 covering KMP, Kotlin Native, AI development, and Gradle.

A Great Gradle Module Structure Ryan Harter outlines a Gradle module structure with separate API and implementation modules for build speed and encapsulation.

Live coding with dir stepper Jake Wharton demonstrates building dir-stepper, a Kotlin/Native tool for managing code steps in live coding presentations.

How We Rewrote Reddit's Video Player on Android Alexey Bykov explains how Reddit built ExoKit, an opinionated video player abstraction optimizing performance through pooling and component design.

The Complete Guide to Android CLI: A Faster Way to Build, Debug, and Automate Android Apps Akshay Nandwana demonstrates the Android CLI tool's capabilities for building, debugging, and automating Android apps directly from the terminal.

Fakes vs Mocks in Testing Dhananjay examines when to use mocks versus fakes in testing, with Android examples for DAOs and repositories.

The Complete Guide to Android CLI: A Faster Way to Build, Debug, and Automate Android Apps Akshay Nandwana examines Android CLI, a terminal interface enabling developers and AI agents to build, debug, and automate Android development.

How I Used a QA Agent Workflow to Reduce Manual Testing Pressure James Cullimore outlines a QA workflow using agents to connect manual test cases with existing automation coverage.

How to Correctly Implement Pull-to-Refresh on Mobile Marcin Moskala explains how to implement pull-to-refresh correctly, keeping data visible while refreshing and handling errors gracefully.

What adb Can Teach You About Broken Android IPC Protections James Cullimore explores Android IPC security flaws and demonstrates proper component protection using adb and signature permissions.

Teaching a Kotlin Chess Engine to Debug Itself Gabor Berenyi shows how to architect a Kotlin logic module so AI agents can autonomously test, debug, and fix it.

Seven Claude Code commands, one Kotlin Multiplatform app on Android and iOS Ali Sadeghi walks through building a Kotlin Multiplatform art gallery app using seven Claude Code commands and a design-first pipeline.

Rewriting Claw's HTML parser for over-the-air updates Harsh Shandilya demonstrates migrating an Android app's HTML parser to Kotlin/JS and Zipline for faster over-the-air updates.

Multi-Module Build Economics: Designing API vs. Implementation Boundaries to Survive 50+ Dev Teams Sarveshwar Maheshwari examines Gradle module architecture and API/implementation boundaries for efficient multi-module builds at scale.

Libraries & Code

Renetik Android A Kotlin framework for reactive, MVC-style Android apps with events, properties, stores, and a UI layer.

ticker A Kotlin library providing kernel-aligned wall-clock ticks for Android without threads or drift.

Andy A Kotlin desktop app for managing Android Virtual Devices and debugging emulators.

Compose A11y Scanner A runtime accessibility scanner for Jetpack Compose that detects missing content descriptions, low contrast, and touch target issues.

Speech Android An on-device speech SDK for Android with ASR, TTS, VAD, and noise cancellation using ONNX Runtime.

News

Kotlin Turns 15: Celebrate the Kotlin Effect JetBrains celebrates Kotlin's 15th anniversary with community spotlights, a browser game, and free Hyperskill courses through September.

Videos & Podcasts

Building androidskills.dev - Episode 3 Code with the Italians stream live coding of androidskills.dev and learning Jetpack Compose.

How I Learned to Stop Worrying and Love Value Semantics in Kotlin Marat Akhin explores value semantics in Kotlin, explaining its benefits and practical application without overcomplicating memory concerns.

Peeling back the Layers: A Navigation3 Deep Dive Kotlin by JetBrains explores Navigation3's API surface, NavDisplay overloads, and how to build custom navigation logic.

My Best Practices For Showing Errors In an App Philipp Lackner breaks down when to use snack bars, banners, or full-screen errors for user-friendly error handling.

Google Play PolicyBytes - July 2026 policy updates Google Play covers July 2026 policy updates on age requirements, package registration, AI integrations, and Target API levels.

Talking to terminals (and how they talk back) Jake Wharton explores terminal communication in command-line tools, covering colors, sizing, and output management.

AI-Driven Performance Optimization - Kotzilla MCP Stevdza-San explores performance optimization using Kotzilla MCP.

Update to the latest AI models without redeploying Firebase shows how to swap AI models live using Remote Config without redeploying.

Context Parameters and API Design Alejandro Serrano Mena explores context parameters in Kotlin and their API design implications.

Run, Kotlin, Run! Marc Reichelt explores multiple ways to run Kotlin on different platforms—JVM, native, JavaScript, and WebAssembly.

We Fit a Full Offline Voice Agent Into 1.2 GB on Android Ivan Potapov demonstrates packing an offline voice-agent pipeline into 1.2 GB on Android.

Follow a step-by-step walkthrough of Dijkstra's algorithm as it discovers the cheapest route through a weighted graph. Includes interactive diagrams, why BFS fails on weighted graphs, and a complete JavaScript implementation.

Here a simple scenario: You have a StringComparer and you want to add your own comparer to the mix - for example a "natural sort" order, so "file2" sorts before "file10" instead of after.

With C# 14 we can extend the family and make it feel natural!

GitHub Copilot CLI agentic workflows help developers plan, test, review, use /fleet, and delegate scoped feature work from the terminal with control today.

Every developer has experienced it.

A test passes perfectly on your machine.

It passes again in your local build.

You push your changes.

Minutes later, the CI pipeline fails.

You rerun it.

Now it passes.

Nothing changed.

Welcome to the world of flaky tests.

Flaky tests are one of the biggest productivity killers in modern software development. They waste engineering time, reduce confidence in continuous integration, and eventually train developers to ignore failing builds.

While flaky tests can have many causes, one of the most common is surprisingly simple:

Hidden external dependencies.

Learn about Unit Testing Best Practices

---

What Is a Unit Test?

The word unit has an important meaning.

A unit test should verify one unit of behavior in complete isolation.

If a test depends on something outside that unit, it becomes harder to reproduce, harder to maintain, and more likely to fail unexpectedly.

A reliable unit test should behave the same way:

• On every developer’s computer
• On every CI server
• Every day
• Every month
• Regardless of machine configuration

External dependencies make that much harder.

---

File System Dependencies

Reading or writing files seems harmless.

Perhaps your test loads a configuration file.

Maybe it writes temporary output.

Or perhaps it reads sample JSON from disk.

The problem isn’t the file itself.

The problem is everything surrounding it.

Questions suddenly appear:

• Does the file exist?
• Is the path correct?
• Is the working directory different on CI?
• Does another test modify the same file?
• Does the operating system lock the file?

None of these questions relate to the business logic you’re trying to verify.

Your test has stopped testing your code and started testing your environment.

---

Network Dependencies

Nothing makes a unit test less predictable than relying on a network.

Even calls to localhost introduce unnecessary risk.

Network-dependent tests can fail because:

• DNS changes
• Firewalls
• VPN connections
• Internet outages
• Service latency
• Authentication failures

The production code may be perfectly correct.

The network simply wasn’t.

Unit tests should never require a network connection unless they are intentionally integration tests.

---

Registry Access

Windows applications often read configuration from the Registry.

That works perfectly in production.

It usually doesn’t belong in a unit test.

Registry values differ between:

• Developer machines
• Build agents
• Customer environments
• Operating system versions

Tests that quietly depend on Registry values become extremely difficult to reproduce.

---

Current Time

Time is one of the most overlooked dependencies.

Consider code like this:

if (DateTime.Now.Hour > 18)

It seems innocent.

Until the test suddenly fails tomorrow.

Or next month.

Or during daylight saving time.

Time-based logic should be isolated so tests can control it.

Otherwise your test suite changes behavior simply because the clock moved forward.

---

Environment Variables

Environment variables are another hidden dependency.

Perhaps your code reads:

• API keys
• Machine names
• Build configuration
• User profile
• Temporary folders

Those values differ everywhere.

Tests that depend on environment variables often pass on one machine and fail on another.

---

Random Values

Random numbers.

GUID generation.

Temporary file names.

Thread scheduling.

These all introduce uncertainty.

A good unit test should produce the same result every single time it executes.

Anything random makes failures harder to reproduce.

---

Why These Dependencies Matter

Individually, these dependencies seem small.

Together they create enormous maintenance costs.

Developers begin saying things like:

“Just rerun the build.”

“That test always fails.”

“Ignore that warning.”

Those phrases are warning signs.

Once developers stop trusting the test suite, the value of automated testing begins to disappear.

---

Detecting Hidden Dependencies

The challenge is that many developers don’t even realize their tests contain these dependencies.

Reading the source code isn’t always enough.

A file access may happen three libraries deep.

A network call may occur through a helper class.

A registry read may happen inside a framework component.

These behaviors only become visible while the test executes.

That’s why runtime analysis is so valuable.

Instead of asking what the code looks like, runtime analysis asks:

What did this test actually do?

---

Integration Tests Are Different

It’s important to distinguish between unit tests and integration tests.

Integration tests are expected to communicate with databases, web services, queues, file systems, and other external components.

That’s their purpose.

Unit tests have a different goal.

They should isolate business logic from those dependencies.

The problem isn’t accessing external resources.

The problem is doing so unintentionally inside a unit test.

---

Building Tests Developers Can Trust

Reliable unit tests share several characteristics.

They are:

• Fast
• Deterministic
• Independent
• Easy to understand
• Easy to maintain

Removing hidden dependencies is one of the most effective ways to achieve those goals.

Developers spend less time investigating failures.

CI pipelines become more reliable.

Refactoring becomes safer.

Confidence increases.

---

Runtime Analysis Finds What Static Analysis Can’t

Static analysis can detect many useful issues.

But runtime behavior tells a different story.

By observing tests while they execute, it’s possible to identify hidden dependencies that are invisible from source code alone.

TypeMock Test Review performs this runtime analysis and highlights unexpected behaviors such as:

• File system access
• Network communication
• Registry usage
• Environment dependencies
• Time-based behavior

These insights help development teams identify fragile tests before they become flaky builds.

---

Conclusion

The best unit tests don’t just pass.

They pass consistently.

They don’t depend on your laptop.

They don’t depend on today’s date.

They don’t depend on network connectivity.

They don’t depend on files that happen to exist.

They validate one piece of behavior in isolation.

As automated test suites continue to grow, identifying hidden dependencies becomes increasingly important.

Because the goal isn’t simply writing more tests.

It’s building tests developers can trust.

---

Continue Reading

• Why 90% Code Coverage Doesn’t Mean Your Tests Are Good
• AI Can Generate Thousands of Tests. Who Reviews Them?

Learn More

TypeMock Test Review, included in the TypeMock Isolator 9.5 , helps identify hidden runtime dependencies that make automated tests fragile, unreliable, and difficult to maintain.

The post Why Unit Tests Should Never Access Files, Networks, or the Registry appeared first on Typemock.