Every data team eventually hits the same wall. A number on a dashboard gets questioned, nobody can prove who is right, and trust quietly leaks out of the whole system. Good data quality and observability is what keeps that from happening, and over six posts I worked through how to build it, one hard part at a time.

Read on its own, each post solves one problem. Read in order, they follow the path a data team walks as it matures: understand the problem, assess the risk, define the fix, build the controls, monitor for what slips through, and respond calmly when something breaks anyway.

Here is the whole set, with the single idea behind each one and the one thing to do about it.

1. Start by naming the problem

“The dashboard is wrong” is not a diagnosis. It is a feeling, and it hides three different problems: a value that is wrong (quality), a value that is correct but late (reliability), and a change upstream that no rule was watching for (observability). Each one has a different owner and a different fix.

So when a number is disputed, ask three questions before you open a query. Is a value wrong or missing? That is quality. Are the values fine but late? That is reliability. Did something change that no rule saw? That is observability. The label is the fastest way to stop three teams from investigating the same ghost.

The takeaway: name the problem before you touch it, and the fix is already half done.

Read the full post: Data Quality, Data Reliability, and Data Observability: Telling the Three Apart

2. Rank issues by risk, not by row count

Profile a messy table and the errors with the biggest counts grab all the attention. They are almost never the ones that matter. Forty duplicate invoices feeding the revenue number will hurt you far more than twelve thousand harmless trailing spaces.

The fix is two columns, not a new tool. Next to every finding, write who feels it downstream and how often it happens, then sort by those instead of by count. Impact times likelihood is the whole model, and it quietly floats your real priorities to the top while the loud, harmless piles sink.

The takeaway: the most dangerous issue is rarely the one with the biggest number.

Read the full post: Count Is Not Risk: Ranking Data Issues by What They Actually Cost

3. Turn “clean” into a rule you can test

“Just make the data clean” cannot be tested, so it can never be met. A usable requirement has three parts: a rule a check can answer yes or no, a threshold that says how much deviation is acceptable, and a named owner accountable for the definition. That is the difference between a wish and a data contract.

Say each expectation out loud and hold it against three questions. Can I write it as a check that returns a number? Have I decided what number passes? Can I name the person who owns it? Three yeses and you have a requirement. A shrug on any of them and you have a preference.

The takeaway: if it cannot fail a test, it is not a requirement.

Read the full post: “Clean Data” Is Not a Requirement: Writing Rules People Can Act On

4. Put controls where they can actually stop the flow

You can have the right check and still catch the problem too late. A validation that runs after publication is a smoke detector wired to go off the morning after the fire.

There are only three places a control can live, and each catches something different. Ingestion stops bad rows at the front door. Transformation catches the errors your own joins and calculations create. The release gate is the last block before anyone downstream reads the data. Put the control at the earliest point that can still stop the problem, decide whether it blocks or only warns, and quarantine what fails so the good data keeps moving.

The takeaway: a check that cannot stop the flow is just a log line.

Read the full post: Where Should a Data Quality Check Live? Gates, Controls, and Quarantine

5. Watch the signals your tests will never catch

Row-level rules only catch the failures you already imagined. They cannot see a feed that arrived stale, a row count that collapsed, a column that changed type, or a value mix that quietly shifted. Those four signals, freshness, volume, schema, and distribution, are where the surprises live.

You cannot spot abnormal without first defining normal. For your most important table, write down four things: when it should arrive, how many rows is normal, what its schema should be, and the usual mix of values in the column you care about. That is a baseline, and a baseline is most of monitoring. Then alert on the size of the change, not the fact of it, so a two percent wobble stays quiet and a sixty percent drop wakes someone.

The takeaway: rules protect you from the mistakes you imagined, monitoring protects you from the ones you did not.

Read the full post: It Passed Every Test and Still Broke: Four Signals Validation Never Catches

6. Respond to a data incident without panic

A wrong number in a live meeting makes everyone want to start changing things immediately. That instinct is how a one-hour fix becomes a three-day cleanup.

Triage first, because you cannot size a response until you know the blast radius. Contain before you fix, because cleaning while the tap runs is endless. Trace to the source instead of patching the number, or it comes back next week. Verify the fix, tell the people who trusted the bad value, and close with a short blameless review so the same break does not return.

The takeaway: changing the data is the middle of an incident, not the end.

Read the full post: You Just Found Bad Data in Production. Now What?

How the six fit together

Read as a set, they build on each other, because you cannot rank a problem you have not named, define a fix for one you have not ranked, or control one you have not defined. Understanding comes first, response comes last, and monitoring is the quiet layer that keeps the whole thing honest between incidents.

Here is the entire guide in one line each, if you ever need to find the right idea fast.

When you hit this	Do this
The dashboard is “wrong”	Name it: quality, reliability, or observability
A long list of data errors	Rank by impact and likelihood, not by count
“Just make it clean”	Write a rule, a threshold, and a named owner
A check that fires too late	Put controls at ingestion, transformation, and the gate
Every test passed, it still broke	Monitor freshness, volume, schema, and distribution
A wrong number in production	Triage, contain, trace, verify, tell, review

Start at the top, or jump to whichever problem is on fire today. Each post also carries a link to its companion Pluralsight course if you would rather see the idea built hands-on.

In short

Data quality and observability is not one skill but six connected moves. Name the problem before you chase it, because “the dashboard is wrong” hides three different failures: a wrong value, a late value, and a silent change. Rank issues by impact and likelihood, not by row count. Turn vague requests like “make it clean” into a rule, a threshold, and an owner. Place controls where they can stop bad data, not merely log it. Monitor the signals row-level tests miss: freshness, volume, schema, and distribution. And when something breaks anyway, respond calmly rather than fast. Read the six posts in order and you have a working playbook for trustworthy data, from the first disputed number to the last blameless review.

Reference: Pinal Dave (https://blog.sqlauthority.com/), X

First appeared on Data Quality and Observability: A Field Guide in Six Parts

Get caught up on the latest technology and startup news from the past week. Here are the most popular stories on GeekWire for the week of July 12, 2026.

Sign up to receive these updates every Sunday in your inbox by subscribing to our GeekWire Weekly email newsletter.

Most popular stories on GeekWire

Seattle indie hit ‘Stardew Valley’ is coming to ‘Magic: The Gathering’

Stardew Valley, the blockbuster farming sim from solo Seattle developer Eric “ConcernedApe” Barone, is coming to Magic: The Gathering via a three-part Secret Lair drop from Renton-based Wizards of the Coast — with original pixel art from Barone himself. … Read More

Tech Moves: Former Amazon exec joins F5; Microsoft security CVP departs; Qualtrics adds leadership

F5 names former Amazon executive as its chief people officer; Rudra Mitra departs as a Microsoft CVP after more than 27 years; and Qualtrics adds leaders. … Read More

AWS EC2 and AI leader Dave Brown to exit, replaced by Amazon exec and Microsoft vet Dave Treadwell

Dave Brown is leaving Amazon Web Services after nearly 19 years, departing at the end of July for a new role outside the company. … Read More

Grunge meets slop: An AI time traveler visits 1992 Seattle when music, not tech, ruled the city

The AI-generated video is an interesting study in how technology that’s very much being built and hyped in Seattle can be used to illustrate what the city sort of looked like more than three decades ago. … Read More

Apptio co-founders reunite to launch enterprise AI startup Thira with $21M in funding led by Madrona

Apptio co-founders Sunny Gupta and Kurt Shintaffer launched Thira, a Bellevue-based enterprise AI startup, with $21 million in seed funding led by Madrona. … Read More

JPMorgan Chase bets on Seattle to build its AI control layer

JPMorgan Chase is building out a new AI software infrastructure team, with a major presence in Seattle, focused on running AI across its data centers and outside providers in a way that controls costs, protects its intellectual property, and avoids tying its fortunes to any one vendor. … Read More

Tech Moves: Remitly CMO departs; Temporal names EVP; Veeam and Qualtrics leadership changes

Rina Hahn leaves role as Remitly’s CMO; Temporal promotes Preeti Somal to EVP amid reorganization; and Veeam and Qualtrics make leadership changes. … Read More

Departing AWS exec Dave Brown is reportedly joining Meta, as Facebook parent mulls its own cloud

Dave Brown, the senior AWS executive whose departure Amazon announced this week, is joining Meta to work on its data center build-out, according to a Wall Street Journal report. … Read More

As General Fusion makes historic Nasdaq debut, report shows global funding surged to $4.5B

General Fusion’s stock is trading up after it became the first fusion energy company to go public on a major exchange, debuting Monday on Nasdaq. … Read More

Venture funding drops in Seattle area as AI boom reshapes startup world

Seattle-area startups raised $2.7 billion in the first half of 2026, down 40% from a year ago, even as U.S. … Read More

Current AI, a non-profit building AI that leaves no one culture behind, has made remarkable progress across devices, AI chat and more.

Elizabeth Stone is the Chief Product and Technology Officer (CPTO) at Netflix, where she oversees Engineering, Product, and Design. Since her first appearance on the podcast two years ago—which remained my second-most-popular episode for more than a year—she has expanded her role to lead product, in addition to engineering. Before Netflix, Elizabeth was VP of Science at Lyft, Chief Operating Officer at Nuna, an economist at Analysis Group, and a trader at Merrill Lynch.

In our in-depth conversation, we discuss:

1. Why “systems thinking” is now the most important skill she looks for

2. How to manage the flood of AI-generated output without losing quality or signal

3. How Netflix thinks about AI fluency as a universal expectation rather than a level-specific skill

4. What “excellence as an operating system” means

—

Brought to you by:

WorkOS—Make your app enterprise-ready, with SSO, SCIM, RBAC, and more: https://workos.com/lenny

Mercury—Radically different banking, now with Command: https://mercury.com/command?utm_source=lennys&utm_medium=sponsored_newsletter&utm_campaign=26q3_brand_campaign

—

Episode transcript: https://www.lennysnewsletter.com/p/netflix-cpto-on-ai-and-the-future

—

Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0

—

Where to find Elizabeth Stone:

• LinkedIn: https://www.linkedin.com/in/elizabeth-stone-608a754

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction

(02:25) AI and role confusion: the storming phase before the forming phase

(07:36) How roles have changed in the past two and a half years

(11:55) Will functions survive? The case for craft specialism

(13:26) What Netflix is hiring more of—and less of

(17:22) Why systems thinking is the rising skill across every function

(20:20) Is the design process dead?

(22:08) Skills trending down

(28:33) AI fluency and Netflix’s career ladder overlay

(31:00) AI use cases beyond coding

(35:12) Netflix’s AI history

(38:36) Excellence as an operating system

(41:11) The pillars of the excellence OS

(46:41) The keeper’s test—and why it’s mostly a positive conversation

(50:21) Attracting top talent in the age of frontier AI labs

(52:54) Junior talent, craft mastery, and the mentorship question

(56:25) Where engineering goes in 5 to 10 years

(59:45) The future of entertainment: beyond film and TV

(1:02:18) AI in Hollywood: Netflix’s creator-enablement position

(1:06:15) Lightning round and final thoughts

—

Referenced:

• How Netflix builds a culture of excellence | Elizabeth Stone (CTO): https://www.lennysnewsletter.com/p/how-netflix-builds-a-culture-of-excellence

• Brian Chesky’s new playbook: https://www.lennysnewsletter.com/p/brian-cheskys-contrarian-approach

• The design process is dead. Here’s what’s replacing it. | Jenny Wen (head of design at Claude): https://www.lennysnewsletter.com/p/the-design-process-is-dead

• Claude Code: https://www.anthropic.com/product/claude-code

• Claude Cowork: https://www.anthropic.com/product/claude-cowork

• Netflix’s “Keeper Test” and Why You Need It | Lorne Rubis: https://www.highlights.lornerubis.com/2015/08/the-netflix-keeper-test-and-the-courage-to-take-it

• Innovation for Filmmaking, By Filmmakers: Why InterPositive Is Joining Netflix: https://about.netflix.com/en/news/why-interpositive-is-joining-netflix

• InterPositive: https://weareinterpositive.com

• Netflix Prize: https://en.wikipedia.org/wiki/Netflix_Prize

• Quarterback on Netflix: https://www.netflix.com/title/81482895

• The Bill Simmons Podcast on Netflix: https://www.netflix.com/title/82186214

• Spencer Pratt on Instagram: https://www.instagram.com/spencerpratt

• Salman Rushdie’s Substack: https://salmanrushdie.substack.com

• Remarkably Bright Creatures on Netflix: https://www.netflix.com/title/81911351

• Eight Sleep: https://www.eightsleep.com

• Tour de France: https://www.letour.fr/en

—

Recommended books:

• Thinking in Systems: https://www.amazon.com/Thinking-Systems-Donella-H-Meadows/dp/1603580557

• Into Thin Air: A Personal Account of the Mt. Everest Disaster: https://www.amazon.com/Into-Thin-Air-Personal-Disaster/dp/0385494785

• Liar’s Poker: https://www.amazon.com/Liars-Poker-Norton-Paperback-Michael/dp/039333869X

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

To hear more, visit www.lennysnewsletter.com



Download audio: https://api.substack.com/feed/podcast/205675851/497b10441135a842dbe35a6891de72bb.mp3

Articles & Tutorials

Sponsored

Advertise to more than 80k Android developers! We reach out to more than 80k Android developers around the world, every week, through our email newsletter and social media channels. Advertise your Android development related service or product!

Change the Map: Feature Flags and Remote Config in Kotlin Multiplatform KMP Bits walks through feature flag and remote config patterns for safely rolling out features in Kotlin Multiplatform.

Test & Set Jesse Wilson explains how testAndSet() simplifies concurrent code by separating business logic from concurrency logic.

Our Droidcon US 2026 Picks Sam Hill highlights must-watch talks from Droidcon US 2026 covering KMP, Kotlin Native, AI development, and Gradle.

A Great Gradle Module Structure Ryan Harter outlines a Gradle module structure with separate API and implementation modules for build speed and encapsulation.

Live coding with dir stepper Jake Wharton demonstrates building dir-stepper, a Kotlin/Native tool for managing code steps in live coding presentations.

How We Rewrote Reddit's Video Player on Android Alexey Bykov explains how Reddit built ExoKit, an opinionated video player abstraction optimizing performance through pooling and component design.

The Complete Guide to Android CLI: A Faster Way to Build, Debug, and Automate Android Apps Akshay Nandwana demonstrates the Android CLI tool's capabilities for building, debugging, and automating Android apps directly from the terminal.

Fakes vs Mocks in Testing Dhananjay examines when to use mocks versus fakes in testing, with Android examples for DAOs and repositories.

The Complete Guide to Android CLI: A Faster Way to Build, Debug, and Automate Android Apps Akshay Nandwana examines Android CLI, a terminal interface enabling developers and AI agents to build, debug, and automate Android development.

How I Used a QA Agent Workflow to Reduce Manual Testing Pressure James Cullimore outlines a QA workflow using agents to connect manual test cases with existing automation coverage.

How to Correctly Implement Pull-to-Refresh on Mobile Marcin Moskala explains how to implement pull-to-refresh correctly, keeping data visible while refreshing and handling errors gracefully.

What adb Can Teach You About Broken Android IPC Protections James Cullimore explores Android IPC security flaws and demonstrates proper component protection using adb and signature permissions.

Teaching a Kotlin Chess Engine to Debug Itself Gabor Berenyi shows how to architect a Kotlin logic module so AI agents can autonomously test, debug, and fix it.

Seven Claude Code commands, one Kotlin Multiplatform app on Android and iOS Ali Sadeghi walks through building a Kotlin Multiplatform art gallery app using seven Claude Code commands and a design-first pipeline.

Rewriting Claw's HTML parser for over-the-air updates Harsh Shandilya demonstrates migrating an Android app's HTML parser to Kotlin/JS and Zipline for faster over-the-air updates.

Multi-Module Build Economics: Designing API vs. Implementation Boundaries to Survive 50+ Dev Teams Sarveshwar Maheshwari examines Gradle module architecture and API/implementation boundaries for efficient multi-module builds at scale.

Libraries & Code

Renetik Android A Kotlin framework for reactive, MVC-style Android apps with events, properties, stores, and a UI layer.

ticker A Kotlin library providing kernel-aligned wall-clock ticks for Android without threads or drift.

Andy A Kotlin desktop app for managing Android Virtual Devices and debugging emulators.

Compose A11y Scanner A runtime accessibility scanner for Jetpack Compose that detects missing content descriptions, low contrast, and touch target issues.

Speech Android An on-device speech SDK for Android with ASR, TTS, VAD, and noise cancellation using ONNX Runtime.

News

Kotlin Turns 15: Celebrate the Kotlin Effect JetBrains celebrates Kotlin's 15th anniversary with community spotlights, a browser game, and free Hyperskill courses through September.

Videos & Podcasts

Building androidskills.dev - Episode 3 Code with the Italians stream live coding of androidskills.dev and learning Jetpack Compose.

How I Learned to Stop Worrying and Love Value Semantics in Kotlin Marat Akhin explores value semantics in Kotlin, explaining its benefits and practical application without overcomplicating memory concerns.

Peeling back the Layers: A Navigation3 Deep Dive Kotlin by JetBrains explores Navigation3's API surface, NavDisplay overloads, and how to build custom navigation logic.

My Best Practices For Showing Errors In an App Philipp Lackner breaks down when to use snack bars, banners, or full-screen errors for user-friendly error handling.

Google Play PolicyBytes - July 2026 policy updates Google Play covers July 2026 policy updates on age requirements, package registration, AI integrations, and Target API levels.

Talking to terminals (and how they talk back) Jake Wharton explores terminal communication in command-line tools, covering colors, sizing, and output management.

AI-Driven Performance Optimization - Kotzilla MCP Stevdza-San explores performance optimization using Kotzilla MCP.

Update to the latest AI models without redeploying Firebase shows how to swap AI models live using Remote Config without redeploying.

Context Parameters and API Design Alejandro Serrano Mena explores context parameters in Kotlin and their API design implications.

Run, Kotlin, Run! Marc Reichelt explores multiple ways to run Kotlin on different platforms—JVM, native, JavaScript, and WebAssembly.

We Fit a Full Offline Voice Agent Into 1.2 GB on Android Ivan Potapov demonstrates packing an offline voice-agent pipeline into 1.2 GB on Android.

Follow a step-by-step walkthrough of Dijkstra's algorithm as it discovers the cheapest route through a weighted graph. Includes interactive diagrams, why BFS fails on weighted graphs, and a complete JavaScript implementation.