Although ‘Big Data’ (and more specifically, the analysis of it) offers several benefits, organizations often fail to derive the correct insights – the data that could help them make better business decisions.

So, in this article, I’ll reveal exactly how organizations can take full advantage of these large datasets to make those better business decisions and improve operational effectiveness.

Before we begin, though, we must first understand the actual problem with big data.

What’s the problem with big data?

Data has quickly become one of the most valuable assets to today’s businesses. Unfortunately, because of the complexity of working with large volumes of data, many organizations still lack a well-defined plan or strategy for harnessing and managing their data.

This is despite many businesses investing billions over the last decade or so in building big data platforms, enabling them to produce significant amounts of data. They’ve acheived that part, but simply haven’t put enough thought into the strategy behind it – so they’re not gaining the valuable insights from the data that would be helping them create meaningful business outcomes.

Organizations should be taking full advantage of big data – using it to generate business value by aligning their big data efforts with specific business objectives.

What is big data?

Before we start looking at the correct ways organizations should be using big data, it’s first important to fully understand what big data actually is.

Big data refers to a collection of structured or unstructured data that traditional information systems cannot handle. It’s often summarized by the ‘6 V’s of Big Data’, which comprises of:

The 6 key benefits of big data

Big data offers some key benefits, including:

The enablement of smarter decision-making

The insights generated from real-time analysis of big data can help organizations make better, faster, evidence-based decisions.

Accelerated development cycles

Technology has increased the importance of shortening product development cycles. To help with this, organizations can use real-time big data and feedback to ensure products are built to meet the expectations of their customers.

Better market research

Real-time analysis of vast amounts of data allows businesses to understand demand, interests, and behavior patterns far more effectively than traditional data collection methods.

More efficient risk management

Businesses can use the big data collected from multiple sources and perform analytics on those datasets to forecast – and prepare for – potential issues in the future.

Increased productivity

Businesses can utilize large amounts of information from multiple sources to review their processes, resources, and workflows. They can spot which areas of the business are performing well, which areas are struggling, and determine what action they may need to take.

Enhanced operational efficiency

Businesses can use large amounts of information from various sources to identify potential bottlenecks and determine better ways to allocate, or use, their resources to resolve these issues. They can also use big data for proactive monitoring of their systems.

5 reasons big data projects fail

This also refers to big data not being fully utlilized by organizations (or organizations not getting the full benefits of using big data.)

Misaligned strategy

While technology is an important enabler of success for big data initiatives within an organization, one of the prime reasons they fail is simply due to misaligned strategy. For example, organizations often start with a technology platform and then try to fit their strategy around it.

Instead, they should define their particular business problem, and then select the best technology platform to help tackle that problem.

Poor data quality

Poor data quality is another concern, as it may further worsen the disconnect between the business and the insights derived from the data. This is usually due to the lack of an established, adhered-to data engineering standard.

Ultimately, data quality should be a continuous discipline. Inaccurate data can be worse than having no data at all!

Treating data as an IT issue

Organizations often make the mistake of treating data as an IT issue, which it is not. When business stakeholders don’t participate in developing a strategic approach to use the data, how do the insights produced from the data match up with the business objectives? The answer is: they don’t.

Complexity

Big data initiatives can fail simply because of complexity. Teams often over-engineer real-time streaming capabilities when they could simply use batch processing that delivers incremental business value.

Lack of trained personnel

To be successful in your big data initiative, you need the right people, in the right place, at the right time. Your organization must have data experts with domain expertise. However, finding the right talent to meet your specific big data needs can, unfortunately, be quite a challenge in itself.

How to get the most out of big data: 5 key strategies

Organizations must know how best to collect, store, and analyze their big data to realize the full value of it. How do they go about doing so? In this section, we’ll look at how to successfully implement big data in an organization and leverage its benefits to drive business value.

Foster collaboration to drive innovation

Creating a culture of collaboration is imperative. Your organization should encourage collaboration to drive innovation, foster exchange of ideas, and facilitate development and learning opportunities among employees.

Focus on the quality of the data

Data quality is extremely important in any big data initiative. The data must be accurate, reliable and up-to-date to ensure the best insights are extracted from it. Hence, you should have proper data management processes in place to ensure you capture, maintain, and monitor the accuracy, reliability, and timeliness of your data.

Select the right method to access big data

First off, an organization should determine the right strategy for accessing big data. Since each organization will have its own unique requirements, use cases, and infrastructure, an organization must decide which approach (or combination of approaches) should be used.

Aggregating data

There are many data sources available to you, and they vary widely in type. It is important to successfully aggregate data by combining and formatting diverse sets into a cohesive whole.

It is actually a challenge for an organization to create a single unified dataset from a variety of fragmented or differently formatted datasets. This has emerged as the most important factor in determining whether your organization will succeed in leveraging the benefits of big data in today’s rapidly changing digital landscape.

Cost management

Without appropriate governance, big data platforms can get very expensive very quickly. Over time, storage, compute, and data transfer costs can increase – often with no warning or notification.

How to manage costs – 7 key strategies

The latter point is a big one. Here are 7 key strategies you can implement to help manage costs:

What does the future hold for big data?

New and emerging technologies such as edge computing, blockchain, and quantum computing will change how businesses handle and process data.

Here are the key trends to watch out for:

Predictive analytics

Predictive analytics is the process of using past data to gauge the future. It’s a capability often used by retailers, financials, and the healthcare industry to improve their operations while anticipating trends and consumer preferences.

AI/ML (machine learning) integration

Using AI/ML enables the automation complex decision points, opening up new methodologies for analyzing and gaining insights from previously unavailable data.

Cloud computing

In the cloud, businesses can securely host and manage large quantities of data without significant investment. They’re also better positioned to respond to changing market conditions.

Big data: summary and key takeaways

The post Big data: what it is, why projects fail, and how to get value from it appeared first on Simple Talk.

Learn how to implement async validation in Angular Signal Forms using the validateHttp function, for example to check whether a username is already taken.

Signal Forms create a form from a writable signal model, where the model is the source of truth, and any change in the form or the model is automatically reflected in the other.

Read more about Signal Forms here.

In this article, we will explore async validation in Signal Forms. You perform async validation for various purposes, such as to:

• Check if usernames or emails are already taken
• Look up data in the database to confirm values
• Use external services (APIs) to verify things like addresses or tax IDs
• Apply business rules that only the server can validate

When creating a signup form, you need to check whether the email is already in use by calling a database via an API. Since this takes time, you should use Signal Forms async validation to handle it.

To validate against an API or HTTP endpoint, Angular provides the validateHttp() function. Let’s explore how to use that.

Create a Signup Form

To begin, create a signup form by first defining an interface to represent the signup type.

export interface SignupFormValue {
  readonly email: string;
  readonly password: string;
}

Next, use it to create the signal-based Signup form as shown in the next code listing:

  signupModel: WritableSignal<SignupFormValue> = signal<SignupFormValue>({
    email: '',
    password: '',
  });

  signupForm = form(this.signupModel);

Then, in the template, bind it to the controls as shown below.

<form (submit)="onSubmit($event)">
  <p>
    <label for="signup-email">Email</label><br />
    <input id="signup-email" type="email" [formField]="signupForm.email" />
  </p>
  <p>
    <label for="signup-password">Password</label><br />
    <input id="signup-password" type="password" [formField]="signupForm.password" />
  </p>
  <p>
    <button type="submit">Sign up</button>
  </p>
</form>

Next, create the submit function as shown below.

protected onSubmit(event: Event): void {
    event.preventDefault();
    submit(this.signupForm, async () => {
      const credentials = this.signupModel();
      console.log('Signup submitted:', credentials);
    });
  }

As of now, we have created the signal-based signup form.

Adding Sync Validations

To make it more usable, let us add some validations to the email field. We are going to add the validations below to the email field.

1. Required
2. Email

  signupForm = form(this.signupModel, (schemaPath) => {
    required(schemaPath.email, { message: 'Email is required' });
    email(schemaPath.email, { message: 'Please enter a valid email address' });
    
  });

Both required and email are synchronous validators, and they run on each value change of controls. The required() validates that the control value is neither null nor an empty string (''), and email() validates the control value against a standard email regular expression pattern.

Both validators execute synchronously on each value change of the control.

We can conditionally show error messages for sync validators in the template using the field-state signals touched(), invalid() and errors().

@if (signupForm.email().touched() && signupForm.email().invalid()) {
    <ul class="error-list">
      @for (error of signupForm.email().errors(); track error) {
        <li>{{ error.message }}</li>
      }
    </ul>
  }

The touched() && invalid() signals are used to show errors only after the user focuses and then blurs the field, so the form does not show errors on initial load.

Adding Async Validations

There are two ways you can add async validation to a signal form.

1. Using validateHttp()
2. Using validateAysnc()

In most cases, applications should use validateHttp() for asynchronous validation, as it simplifies HTTP-based validation with minimal configuration and supports most common scenarios.

validateAsync() is a lower-level API that directly exposes Angular’s resource primitive, providing full control over the validation process but requiring more implementation effort and a deeper understanding of the resource API.

Let’s see how we can use validateHttp() to check whether an email exists or not .

validateHttp(schemaPath.email, {
      request: ({ value }) => {
        const emailValue = value();
        if (!emailValue) return undefined; // skip when empty
        return `http://localhost:3000/user/check?email=${emailValue}`;
      },
      onSuccess: (response: { exists: boolean }) => {
        return response.exists
          ? {
              kind: 'emailTaken',
              message: 'This email is already registered',
            }
          : null;
      },
      onError: () => ({
        kind: 'serverError',
        message: 'Could not verify email. Please try again later.',
      }),
    });

We have an API that returns a response indicating whether the email already exists in the database, and we use the response’s exists property to determine whether the email is already registered.

{
  "exists": true,
  "user": {
    "id": 1,
    "name": "User 1",
    "email": "user1@example.com"
  }
}

We can show the pending state and error message from both sync and async validator as shown below:

@if (signupForm.email().pending()) {
    <span class="pending">Checking email availability...</span>
  }
  @if (signupForm.email().touched() && signupForm.email().invalid()) {
    <ul class="error-list">
      @for (error of signupForm.email().errors(); track error) {
        <li>{{ error.message }}</li>
      }
    </ul>
  }

The validation flow works like this:

• User enters a value.
• Synchronous validators run first.
• If sync validation fails, async validation does not run.
• If sync validation passes, async validation starts, and pending() becomes true.
• Once the request completes, pending() becomes false.
• Errors are updated based on the response.

This can be put in a flow chart as shown below:

Putting everything together, a signal-based signup form with both synchronous and asynchronous validators will look like this:

signupModel: WritableSignal<SignupFormValue> = signal<SignupFormValue>({
    email: '',
    password: '',
  });

  signupForm = form(this.signupModel, (schemaPath) => {
    required(schemaPath.email, { message: 'Email is required' });
    email(schemaPath.email, { message: 'Please enter a valid email address' });
    validateHttp(schemaPath.email, {
      request: ({ value }) => {
        const emailValue = value();
        if (!emailValue) return undefined; // skip when empty
        return `http://localhost:3000/user/check?email=${emailValue}`;
      },
      onSuccess: (response: { exists: boolean }) => {
        return response.exists
          ? {
              kind: 'emailTaken',
              message: 'This email is already registered',
            }
          : null;
      },
      onError: () => ({
        kind: 'serverError',
        message: 'Could not verify email. Please try again later.',
      }),
    });
    
  });

  protected onSubmit(event: Event): void {
    event.preventDefault();
    submit(this.signupForm, async () => {
      const credentials = this.signupModel();
      console.log('Signup submitted:', credentials);
    });
  }

And on the template, the pending state and error messages can be shown as below:

<form (submit)="onSubmit($event)">
  <p>
    <label for="signup-email">Email</label><br />
    <input id="signup-email" type="email" [formField]="signupForm.email" />

  </p>
  @if (signupForm.email().pending()) {
    <span class="pending">Checking email availability...</span>
  }
  @if (signupForm.email().touched() && signupForm.email().invalid()) {
    <ul class="error-list">
      @for (error of signupForm.email().errors(); track error) {
        <li>{{ error.message }}</li>
      }
    </ul>
  }
  <p>
    <label for="signup-password">Password</label><br />
    <input id="signup-password" type="password" [formField]="signupForm.password" />
  </p>
  <p>
    <button type="submit">Sign up</button>
  </p>
</form>

validateHttp Function

The validateHttp function contains three components:

1. Request function
2. Success handler function
3. Error handler function

In the validateHttp, we have set the request function. It returns one of these three values:

1. Undefined to skip the validation.
2. A URL for performing the validation using the GET operation.
3. An HttpResourceRequest if you need to set additional request options, such as the verb header, etc. This is the same object you use with httpResource to configure the request object.

Read more about the httpResource here.

The onSuccess function receives the HTTP response and returns validation errors or undefined for valid values. It can also return multiple errors when needed.

The onError function handles request failures like network errors or HTTP errors.

If you need to validate an email using an endpoint with a POST request, custom headers and the email in the request body, you can do so by returning an httpResourceRequest from validateHttp, as shown below.

signupForm = form(this.signupModel, (schemaPath) => {
    required(schemaPath.email, { message: 'Email is required' });
    email(schemaPath.email, { message: 'Please enter a valid email address' });

    validateHttp(schemaPath.email, {
      request: ({ value }) => {
        const emailValue = value();
        if (!emailValue) return undefined;
        return {
          url: 'http://localhost:3000/user/check',
          method: 'POST',
          body: { email: emailValue },
          headers: { 'x-api-key': 'my-secret-key' },
        };
      },
      onSuccess: (response: { exists: boolean }) => {
        return response.exists
          ? { kind: 'emailTaken', message: 'This email is already registered' }
          : null;
      },
      onError: () => ({
        kind: 'serverError',
        message: 'Could not verify email. Please try again later.',
      }),
    });
  });

The other function for async validation is validateAsync() function. It is a lower-level API that exposes Angular’s resource primitive directly. It offers complete control but requires more code and familiarity with Angular’s resource API. Most applications should use validateHttp().

You should use validateAsync() for these purposes:

• Non-HTTP validation – WebSocket connections, IndexedDB lookups or Web Worker computations
• Custom caching strategies – Application-specific caching beyond simple memorization
• Complex retry logic – Custom backoff strategies or conditional retries
• Direct resource access – When you need the full resource lifecycle

In a future article, we’ll take a deeper look at validateAsync. For now, I hope this gives you a clear understanding of how to implement async validation in Angular Signal Forms using the validateHttp function. Thanks for reading!

When a TypeScript codebase grows, you usually start organizing it into feature folders. Before long, each folder contains both the functions the rest of the application should use and the helpers that only exist to support them.

Clear boundaries help you stay in control, avoid obvious bugs, and make larger changes with more confidence. This matters even more when AI can make broad changes under the hood. You still need to preserve the functionality that the rest of the application relies on.

Although the examples use TypeScript, the same ideas work for JavaScript. Workspaces, package exports, barrel files, and the Biome and ESLint rules are not TypeScript-only. The TypeScript-only parts are project references, declaration output, and type-aware ESLint configuration.

In this article, we'll look at package exports in a monorepo and lint rules for a single codebase.

Packages And Workspaces

The usual way to create a stronger boundary is to split a monorepo into packages. Each package has its own package.json, build output, and public entry points. Other packages then depend on it by name instead of reaching into its source tree.

pnpm workspaces find and link the packages locally. They do not decide which files are public, though:

# pnpm-workspace.yaml
packages:
  - apps/*
  - packages/*

The package's exports map says what other packages are allowed to import:

// packages/orders/package.json
{
  "name": "@acme/orders",
  "private": true,
  "types": "./dist/index.d.ts",
  "exports": "./dist/index.js"
}

The source entry point lists the symbols that belong to that public API:

// packages/orders/src/index.ts
export { createOrder } from "./create-order.js";
export type { Order, OrderTotals } from "./types.js";

The consuming app can import createOrder from @acme/orders, but this deep import is not part of the public API:

import { calculateTotals } from "@acme/orders/calculate-totals";

With a module resolution mode that understands package exports, TypeScript rejects the deep import because that subpath was never exported. It's worth noting that a path alias or relative import into another package's src directory can still bypass this, so teams usually lint against cross-package source imports too.

That is the important distinction: the workspace links packages together, while the exports map controls which package paths consumers can use. TypeScript project references help with build ordering and keep the projects separate:

// packages/orders/tsconfig.json
{
  "compilerOptions": {
    "composite": true,
    "declaration": true,
    "outDir": "dist",
    "rootDir": "src"
  },
  "include": ["src"]
}

Project references are TypeScript-specific. You can read more in TypeScript's project references documentation and Node's exports documentation.

When Everything Is One Codebase

Packages are useful when a repository already has clear package-sized parts. They are more ceremony than many applications need, though. If everything lives under one src directory, there is no package boundary between orders and checkout:

src/
├── features/
│   ├── orders/
│   └── checkout/
└── app/

An export is still importable from any file that can resolve its path. Path aliases make imports nicer, but they do not make folders private. In a single codebase, you need a convention and a lint rule to stop people bypassing public entry points.

Barrel Files As A Convention

A barrel file is usually an index.ts file that re-exports the symbols meant for use outside a folder:

// src/features/orders/index.ts
export { createOrder } from "./create-order";
export { getOrderSummary } from "./order-summary";

Code elsewhere imports from @/features/orders, while internal helpers stay out of the barrel. Without linting, someone can still import them directly:

import { calculateTotals } from "@/features/orders/calculate-totals";

Enforcing Barrel Imports

ESLint's import/no-internal-modules rule can reject submodule imports while still allowing the folder entry point:

{
  "rules": {
    "import/no-internal-modules": [
      "error",
      { "forbid": ["@/features/*/*"] }
    ]
  }
}

The exact pattern depends on your aliases. For larger architectures, eslint-plugin-boundaries can define which entry points different kinds of folders may use.

Biome does not require barrel imports. Its noBarrelFile rule does the opposite and reports files that re-export other modules.

The Limits Of Barrel Files

Barrel files are useful documentation, but they also have a few problems:

• They are only a convention without linting. An index.ts file is not a visibility boundary, so deep imports are still possible.
• They can make the dependency graph bigger. Re-exporting runtime modules can load or analyze more modules than the consumer needs. This is the reason Biome provides noBarrelFile.
• They can run top-level side effects. A runtime re-export adds its target module to the import graph. When the barrel is loaded, that module may be instantiated and evaluated even if the caller needs a different export. Top-level work such as registering a handler, changing global state, reading environment state, or logging can then run unexpectedly. Bundlers may remove unused modules when static analysis and side-effect metadata allow it, but tree-shaking is not a replacement for side-effect-free modules.
• They can hide ownership and create cycles. A large index.ts becomes another API manifest, and importing back through it from an internal file can introduce a cycle.

If you want smaller module graphs and clearer ownership, Biome's no-barrel approach is a good alternative.

No Barrels With Biome

Enable noBarrelFile together with noPrivateImports:

{
  "linter": {
    "rules": {
      "correctness": {
        "noPrivateImports": "error"
      },
      "performance": {
        "noBarrelFile": "error"
      }
    }
  }
}

Now you can import files directly, while visibility is declared on their exports:

// src/features/orders/create-order.ts

/** @public */
export function createOrder() {
  return calculateTotals();
}

/** @package */
export function calculateTotals() {
  // ...
}

Biome recognizes @public, @package, and @private, along with the equivalent @access values. Package visibility allows imports from the same folder and its subfolders. Private exports are normally limited to the same module. index.ts and mod.ts files get special handling so their submodules can use private exports.

Code inside src/features/orders can use calculateTotals, while code in src/features/checkout gets a lint error if it imports it.

Flipping The Default Is Not Available

It would be convenient to make every export package-private and add @public only to the intended API. Biome does not provide that setting. Untagged exports are public, and there is no supported defaultVisibility option.

Mark internal exports with @package or @private, and leave public exports untagged or mark them with @public. noPrivateImports checks static imports, but not dynamic import() or CommonJS require() calls. It also ignores resources and dependencies under node_modules.

The ESLint Alternative

With ESLint, eslint-plugin-import-access provides a similar JSDoc-based model for typed configurations. It supports @package, @private, and @public, and includes a TypeScript language-service plugin that can keep restricted exports out of auto-completion.

For broader architecture rules, eslint-plugin-boundaries can classify folders and control which parts of the codebase may import from each other.

Faster Feedback In VS Code

The command line and CI remain the final authority, but editor extensions can show boundary violations while you work instead of making you wait for a build or full lint run.

Install the official Biome extension with @biomejs/biome, or the Microsoft ESLint extension with the ESLint plugins used by your project:

pnpm add -D @biomejs/biome eslint eslint-plugin-import

If you use eslint-plugin-import-access, add it with typescript and typescript-eslint as required by your ESLint configuration. The extensions provide inline diagnostics and quick fixes:

// .vscode/settings.json
{
  "editor.formatOnSave": true,
  "editor.codeActionsOnSave": {
    "source.fixAll.biome": "explicit",
    "source.fixAll.eslint": "explicit"
  },
  "eslint.validate": [
    "javascript",
    "javascriptreact",
    "typescript",
    "typescriptreact"
  ]
}

Choose one tool to own formatting and let the other focus on lint diagnostics. An invalid import shows up in the editor instead of surprising you after the build or CI lint step.

Summary

In a monorepo, pnpm workspaces connect packages, project references describe the TypeScript build graph, and package exports define the public paths. In a single codebase, barrels are a convention that ESLint can enforce, while Biome can ban them and enforce visibility on individual exports.

Biome keeps untagged exports public, so internal exports must be annotated explicitly.

The same guidance applies to JavaScript. Only the project references, declaration output, and typed-linting parts require TypeScript.

Five weeks ago I wrote that the least glamorous piece of an agent loop is also the one that decides whether it compounds: memory. A markdown file outside the context window that holds what is done, what is next, and what was learned, because the model forgets all of it between runs. Write the memory file before the loop.

What I left open, because there was nothing to point at, was the format. My memory file looked nothing like yours, and neither of our agents could read the other’s.

Three days after that post went live, Google shipped an answer.

The pattern everyone copied

Andrej Karpathy published a gist in April he called the LLM wiki, and it collected thousands of stars and forks. It’s meant to be pasted straight into a coding agent. Instead of indexing your documents for RAG and re-deriving answers from raw text on every query, the agent builds a wiki and keeps it current: interlinked markdown pages, an index.md with a one-line summary per page, a log.md recording every change, entity pages that grow as sources come in. Drop in a meeting transcript and the agent reads it, updates a dozen existing pages, fixes the cross-references, and appends to the log in one pass.

It took off for the same reason wikis usually die. A knowledge base is valuable in exact proportion to the bookkeeping nobody wants to do: summarizing, linking, reconciling contradictions, pruning stale claims. Karpathy’s line, which Google now quotes back in its own announcement, is that LLMs “don’t get bored, don’t forget to update a cross-reference, and can touch 15 files in one pass.” The human curates sources and asks questions. The agent does the janitorial work that made every previous wiki rot.

A wiki only your agent can read

Then everyone built one, and every one is a dialect. Mine links related pages in the frontmatter; yours links them at the bottom of the body. Mine has a tags field; yours calls it categories. None of this matters while the wiki serves one person, because the schema lives in your CLAUDE.md and your agent reads it on every run.

It matters the moment the wiki has to travel. Hand your knowledge base to a teammate and their agent starts guessing at your conventions, or misses half the structure entirely. A platform team that wants one shared wiki, queried independently by everyone’s agents, has no format to agree on. A knowledge base you can’t hand to someone else’s agent is a silo of one. And a team’s collective knowledge should outlive any single person’s markdown habits.

What Google actually shipped: the Open Knowledge Format

On June 12, two tech leads in Google Cloud’s data analytics engineering org announced the Open Knowledge Format, with a spec and reference tooling on GitHub. Strip the branding and OKF v0.1 is a formalization of the Karpathy pattern:

• A bundle is a directory tree of markdown files. Two filenames are reserved: index.md for progressive disclosure and log.md for update history. Everything else is a concept document.
• Every concept carries YAML frontmatter with exactly one required field: type. Five more are recommended: title, description, resource, tags, timestamp. Producers can add anything; consumers must preserve what they don’t understand.
• A markdown link from one concept to another asserts a relationship. The prose around the link says what kind.

The whole spec fits on a page, and that’s deliberate. The announcement names three principles: minimally opinionated; producers and consumers independently swappable; a format rather than a platform. No required SDK, no compression scheme, no new runtime. Reading a bundle is cat; distributing one is git clone. The repo backs it up with reference tooling: an enrichment agent that drafts concept docs from BigQuery datasets, and a static visualizer that renders a bundle as a graph. Three sample bundles, built by that same agent, give you something to copy from.

The critique is the feature

The pushback writes itself: there is almost nothing here. Folder indexes and five suggested frontmatter fields, on top of a pattern the community already had. True. Also the point.

The standards that stick are embarrassingly small. MCP didn’t model your tools; it standardized the socket they plug into. AGENTS.md fixed nothing but a filename, and that was enough for Neo to read the same conventions file as every other agent. Agent Skills gave procedure a place to live, and docs sites settled on /llms.txt for published content. Knowledge was the layer still missing its boring little standard. Whether OKF specifically is the one that survives matters less than the shape it commits to: markdown, frontmatter, git. Every other layer of the agent stack already landed there, so that bet is safe.

Knowledge as code

Platform teams should care about this before anyone else. Infrastructure as code captures the what: the resources, the config, the dependency graph. It has never captured the why. The runbook for rotating credentials, the decision record explaining the pinned CNI version, who owns the cluster, what the incident in November taught you. That knowledge lives in a wiki behind an API, a shared drive, a Slack thread, and the heads of two senior engineers. Google’s announcement frames this fragmentation as the problem: agents reassembling answers from systems that each speak a proprietary format.

OKF’s answer is a move platform engineers will recognize, because it’s the same one that produced IaC: put it in git, make it diffable, review changes in pull requests. Knowledge as code, sitting next to the infrastructure as code it explains:

platform/
├── index.md
├── log.md
├── services/
│ ├── index.md
│ ├── checkout-api.md
│ └── payments-worker.md
├── runbooks/
│ ├── index.md
│ └── rotate-database-credentials.md
└── decisions/
├── index.md
└── why-we-pin-the-vpc-cni-version.md

The recommended resource field takes a URI naming the asset a concept describes, and a Pulumi URN is exactly that:

---
type: runbook
title: Rotate the payments database credentials
description: Zero-downtime credential rotation for the payments Postgres instance.
resource: urn:pulumi:prod::payments::aws:rds/instance:Instance::payments-db
tags: [payments, postgres]
timestamp: 2026-07-01T09:30:00Z
---

Now the runbook names the exact resource it operates on. An agent planning a change to that database can walk from the URN to the runbook to the decision record that explains the constraint, before it proposes anything. In the loop post I said skills are intent written down, the conventions an agent reads instead of guessing. A knowledge bundle is the other half: experience written down. Tools give the loop hands, skills give it habits, and the bundle is what it gets to remember.

Before writing any of that down, I ran the test. A fresh agent got two inputs, the spec and this bundle, and one question: which services touch the payments database, and how do I rotate its credentials without downtime? It read six files and answered correctly, down to why rotating a second time too soon is the dangerous move. What sold me was the file it didn’t read. It left the CNI decision record closed, because the cross-links mark that page as a cluster concern, not a database one.

The bundle is public, exactly the tree above. Clone it, hand your agent the spec, and ask your own questions:

github.com/dirien/pulumi-platform-okf-bundle

What a standard won’t do for you

Three caveats, so nobody mistakes this for magic.

It formats knowledge; it doesn’t create it. The warning from the loop post applies with no edits: memory is what lets a loop compound, and slop compounds right alongside it. A wrong runbook in a beautifully conformant bundle is still a wrong runbook, now served to every agent on the team with confidence. The leverage isn’t the format. It’s that knowledge changes finally go through review like code changes.

v0.1 is a draft, and the name may not survive. Google says so itself: the announcement calls v0.1 “a starting point, not a finished standard.” OKF could lose to a better spec next year. I’d start anyway, because converting markdown with frontmatter into markdown with slightly different frontmatter is the cheapest migration you’ll ever run, and it’s precisely the kind of mechanical pass an agent finishes in an afternoon. The shape is the commitment. The name is a detail.

Conformance is a floor, not hygiene. The spec requires consumers to tolerate broken links, missing indexes, and unknown types. That tolerance keeps readers working, and it also means nothing forces your bundle to stay healthy. Karpathy’s gist included a lint pass for a reason: contradictions, orphaned pages, stale claims. Keep it, and run it the way you lint code, on a schedule, with findings that reach a human.

Where to start

1. Pick the knowledge that already hurts. The runbook nobody can find during an incident, the onboarding doc that’s wrong in ways only one person knows. One domain, not the whole org.
2. Let the agent do the conversion. Paste the spec into your coding agent, point it at the existing folder of docs, and have it refactor them into a bundle. This is exactly the bookkeeping LLMs are good at, and it works as well on a wiki export as on a fresh start.
3. Put the bundle where the code lives. Same repo or a sibling, but in git, behind pull requests. The day a knowledge change gets a review comment is the day the bundle becomes trustworthy.
4. Wire it into the loop. Agents read index.md first and drill down only when a concept earns it, which keeps the context window small. The memory file I told you to write before the loop now has a format, and every agent you run, in whatever harness you built, can read the same one.

The advice from June holds: write the memory file before the loop. What changed is that the file no longer has to be a private dialect. There’s now a one-page, git-native spec for the layer your agents think with, and the smallness that makes it look trivial is the property that lets it spread. The loop does the typing. The wiki does the remembering. And the remembering is now something you can put in a pull request and hand to the next agent, like everything else you ship.

Software engineers are often told to “start playing politics”, but most engineers have no idea what that means.

Their reference point for “playing politics” comes from fiction like Game of Thrones. Are they supposed to raise an army and depose the CEO, or poison each other at team lunch? Should they book Zoom calls with each other and plot schemes? All of that is obviously ridiculous. In terms of Game of Thrones, software engineers are not lords and ladies. We’re the soldiers and workers of the realm. So you should think about “playing politics” in the way a castle guard would, not one of the major players.

The castle guard are not going around poisoning people or forming coalitions between the great powers. They are largely keeping their heads down. But in order to do that, they have to stay aware of the political currents, or they’re liable to do something catastrophically stupid: for instance, making an enemy of a powerful courtier, or arresting somebody who’s on an important mission for the king.

Given that, the basic principles of playing politics are something like this:

• Be aware of who’s powerful and who’s not
• At all costs, avoid making powerful enemies
• Help powerful people as best you can
• Make sure they know you’re helping them (without annoying them)

Be aware of who’s powerful and who’s not

As a software engineer in a large company, you will not be a powerful person. Powerful people are typically in senior management: VPs, directors, and so on¹. However, not everyone in senior management is powerful. Some are killers who have the active support of the CEO, while others are confused incompetents.

How do you know which is which? If someone is clearly ferociously competent, they’re always going to have some power, since upper management tend not to ignore useful tools. But you can’t rely on competence as your only guide. Some managers are powerful for other reasons: they’re friends with the CEO, or they have strong relationships with other groups like legal or sales, or they’re simply willing to do whatever upper management wants done.

One signal is who’s leading the important projects. Read your CEO or CTO’s internal updates and pay attention to the projects that are called out by name. Organizations tend to give key tasks to trusted lieutenants. If a manager is leading an area that’s never under the spotlight, they probably don’t have enough clout.

Another signal is hiring. Is a manager’s team growing or shrinking? Particularly post-ZIRP, headcount is a rare and precious resource. A manager who’s able to get it is likely a powerful manager, or at least is reporting to a powerful director or VP (which often amounts to the same thing).

At all costs, avoid making powerful enemies

First, you should try not to make any enemies at all. Most software engineers who get “playing politics” wrong do it by needlessly alienating people: by being rude, unhelpful, abrasive, making non-technical people feel stupid, and so on. This post isn’t really about that. I’m assuming that you can figure out how to be a generically pleasant person on your own.

However, competent software engineers will make some enemies. If you’re out there making projects happen, some people aren’t going to like the way you do it, and won’t be a fan of any compromise you offer. I wrote about this in Big tech engineers need big egos: the only way to avoid making enemies is to change nothing, but that’s incompatible with doing the job.

Given that, be selective about which enemies you make. If you’re making a technical decision that’s either going to require work from team A or team B, and neither team wants to do it, you should try to pick the team with the least political cover. If you need a powerful VP’s team to do something they won’t like, try to be maximally respectful about it: get that team’s core engineers on-side if you can, or book a meeting with the powerful manager and explain the situation, or (better yet) ask the powerful manager sponsoring your project to go and talk to the other VP for you. (If you don’t have a powerful manager like this, consider abandoning your project).

Give way to powerful managers when at all possible. Every so often you really do have to stand your ground — if the system will truly collapse otherwise, or a major customer will have an incident, or if the technical decision really is entirely bone-headed — but almost all cases are not like this. The best advice I’ve ever gotten about playing politics came from a manager I worked with long ago²:

This is not the hill you want to die on.

When I’m about to pick a fight or say something argumentative, and I’m not 100% convinced it’s necessary, I ask myself: is this the hill I want to die on? And it never is.

The three rules about disagreeing with powerful people are:

• Make sure you do it in private
• Be polite
• When they overrule you, stop arguing immediately

Disagreeing in private rarely hurts, if you follow these rules. In fact, it can help. If you can manage to disagree with a manager, get overruled, and then follow their plan without complaining, that can be the best way to gain a powerful friend. But if they think you’re going to keep griping about it, or worse still, complain to the rest of the team and foment some kind of rebellion, there’s no quicker way to make a powerful enemy.

If you have powerful enemies at a company (for instance, the CTO or an influential VP doesn’t like you), quit. It’s really that bad. I have never seen this situation turn itself around, except in the very rare case where the CTO or VP is already looking for greener pastures and jumps ship. You cannot recover the situation: they have no incentive to give you the chance to change their mind, and they have almost unlimited ability to screw you on promotions, raises and layoffs.

That’s why this piece of advice is second in the list. If you aren’t helpful or if your contributions are invisible, you can work on that and fix it. But if you’ve made powerful enemies, you’re done for.

Help powerful people as best you can

Just as it’s fatal to make powerful enemies, it’s very useful to make powerful friends. How can you do this? Remember you’re a palace guard, not a great lord: you make friends by doing your job. However, you can choose to do your job a little more proactively and diligently when you’re doing it for someone with political clout.

One obvious application of this principle is that you should answer Slack messages from powerful people immediately. If you see an ordinary Slack question pop up while you’re doing some task, it’s okay to get to it when you get to it. In fact, it’s ideal not to respond to all questions immediately, so you don’t set unreasonable expectations (and so you don’t seem like you’re sitting around doing nothing). But when a VP comes in with a question, don’t make them wait: answer the question immediately. If the question requires research, send a “let me look into that right now” message, then do the research. This is the easiest way to get a reputation for being helpful³.

Another way to do this is to lean in on important projects. Suppose you do ten projects in a year. Eight of them are normal, low-priority projects, and two of them are high-profile (say, finishing some big feature before your company’s yearly conference). It’s a mistake to allocate your effort equally to all ten. I wrote about this at length in Doing nothing at work: you should be operating at 80% capacity (or less), so you can then ramp up to 120% when it really matters.

Pay attention to the narrative that powerful people are trying to push. Here are some potential narratives:

• We’ve had a lot of turnover and reorgs lately, but we’re all starting to pull together as a team now
• Isn’t it great how focused we all are on reliability work after last month’s incident?
• The conference this week is the most important thing, so we’re all being very careful not to break anything
• We’re an AI-forward team that’s looking for the best ways we can leverage LLMs into our team processes
• Although this project had a rocky start, we’re now all aligned on the way forward

You don’t necessarily have to jump in and start cheerleading, but you should at least not do anything that you know is going to make the narrative look weak. For example, on that last point, it’s foolish to openly argue that the project really was fine all along. Bring it up privately, not publicly, or you risk ruining some clever piece of propaganda that the manager in question is trying to push on the rest of the organization⁴.

Finally, an underrated way to help powerful people is to offer them social support and information. Slack messages and planning emails might seem unimportant to you, but powerful people often live in that environment: their primary tool is writing messages like these, just like your primary tool is writing code. Reading and responding (in a supportive way) to these messages is something that most engineers don’t bother to do, but it goes a long way.

Likewise, dropping a senior manager a line now and then (say, a heads-up that a particular project landed successfully, or that you got good metrics about some feature) is surprisingly helpful. Senior managers live in an information-poor environment: for them to learn something about a team’s work, that information has to bubble up through several layers of interpretation and summary. In my experience, they’re appreciative of being drip-fed the occasional piece of information, so long as you keep it brief and relatively rare.

Make sure they know you’re helping them

If you’re directly responding to a VP’s Slack messages or DMing them information, they know you’re the one doing it. But if you’re just doing your job and working hard on projects they care about, they might not notice. Being invisible is probably the most common way engineers fail at playing politics.

Fortunately the fix is simple: tell people what you’re doing. If you fix an important bug for a launch, write a message in that launch’s Slack channel saying “hey, I just fixed this bug”. What if you don’t like bragging? Get over it. You have to be comfortable publicly telling people what you’ve done. You should also keep a brag document so you can repeat all of this at review time.

Another, subtler way to do this is to gain the trust and respect of the powerful engineers in your area. Senior managers will always have a few trusted engineers they rely on to assess technical questions. They will ask those engineers what they think about you, and will broadly trust those answers. The good news is that if you’re competent and useful, those engineers will already value you, so you don’t have to do anything special: just be good at your job.

Technical power

Is playing politics all about sucking up to senior managers? Basically, yeah. A less cynical way to describe it would be “aligning with the values of the company”. If you think your company is doing good things, you should want to do that anyway! In any case, what that comes down to is figuring out what the people in charge want, giving it to them, and making sure they see you doing it. However, there’s still some scope to get what you want out of the deal.

I said earlier that software engineers do not wield organizational power. However, that doesn’t mean you’re powerless. Technical ability is a source of real power, if a delicate and unreliable one. The movers and shakers in tech companies are utterly dependent on technical people to implement their vision and to give them clear answers about the system.

There are many subtle ways you can leverage this. One I wrote about in How I influence tech company politics as a staff software engineer is to wait until important people at the company want to do something (say, improve reliability), then offer them a technical plan that does it your way. Another one is to become so useful that you’re actively in demand to lead projects, and then run the project how you want.

You probably won’t be able to change the company’s grand strategy. But how that strategy is implemented has a lot of specific technical detail, and you can put yourself in a position to decide on those details.

Conclusion

Playing politics isn’t about plotting and scheming, and it isn’t just about being a friendly, likeable person (although that helps). It’s about figuring out how your company actually operates: who makes the decisions, who gets consulted, what behavior gets rewarded, and so on. The most basic way to do that is to figure out who is powerful, get out of their way, and (if you can) help them get what they want.

Unmesh Joshi illustrates how LLMs work very effectively with Domain-Specific Languages (DSL). The constrained nature of DSLs make it easier for the LLM to build code based on natural language interactions with the user. LLMs can also act as a brainstorming partner to help develop such a DSL and its underlying Semantic Model. Tickloom is an example of this: a domain model and DSL for illustrating distributed system behavior. Such a DSL can act as the key source of truth for software systems in the world of LLMs

more…