On August 17, GitHub experienced an outage that lasted 7 hours and 47 minutes. It disrupted github.com, authentication, GitHub Actions, APIs, pull requests, issues, and Copilot, affecting developers and organizations around the world. If you were trying to ship software that day, we let you down.

This was our second significant incident in August, following an actions failure on August 6. In March and April, I shared the work underway to improve GitHub’s reliability. We have made progress, but these incidents make clear that we must accelerate this work.

What happened

Our investigation found that the outage began when traffic reached a new peak, and a critical infrastructure component in our Central US data center failed to scale with it. The resulting capacity pressure spread through our systems, causing authentication failures and disrupting multiple GitHub services.

Recovery required several coordinated actions. Teams rerouted traffic, isolated affected infrastructure, and restored services in stages. Most GitHub services recovered earlier that day, but some Copilot services took longer. Errors in those services triggered a client-side retry loop that increased traffic during recovery. We had to mitigate that behavior before we could safely restore traffic. The full root cause analysis includes a detailed technical timeline.

Neither outage was caused by a code or configuration change. Both incidents were capacity failures at their core. We failed to scale critical components before demand exceeded their capacity. Since April, monthly commits have grown from 1.4 billion to 2.9 billion. That growth explains the pressure on our systems, but it does not excuse these outages.

What we have done and what comes next

As part of the reliability commitments we made earlier this year, we have focused on three priorities: adding capacity, improving efficiency, and removing architectural bottlenecks. We have since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity. We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure.

Today, Azure serves roughly 58% of GitHub’s platform load and half of all Git operations, up from 12% of platform load in May. This expanded footprint has also supported the growth in GitHub Actions job runs shown below.

Azure’s infrastructure and capacity have also accelerated our work to scale the largest monorepos. Our next milestone is an architecture that scales read capacity linearly with the number of readers, enabling unlimited read operations. We will roll it out gradually, beginning with the largest monorepos.

Scale is not our only challenge. As the pace and complexity of change increased, our existing operational practices did not keep up. We have redirected teams and resources toward availability and invested in stronger testing, safer rollouts, better observability, and more effective alerting. We have made progress, but this work is not complete.

In addition, we are also isolating critical systems and removing shared dependencies between them. This work is designed to reduce the likelihood of an outage and limit its impact when one occurs.

We learn from every outage and add new work to our availability workstream. The August 6 and August 17 incidents led to two immediate changes. First, we are applying consistent retry limits, retry budgets, and variable timeouts across service-to-service interactions to prevent retry storms and cascading load. Second, we are reviewing lower-priority CPU and memory alerts to identify components that could fail during sudden traffic spikes.

Our commitment to high availability isn’t just a technical promise. The developer community depends on GitHub to build, ship, and operate their work. That is only possible if you can rely on us, and on August 17, you couldn’t. It is our responsibility to fix that. We’ll earn your trust through the scaling and reliability of the platform.

The post The August 17 outage, and the work ahead appeared first on The GitHub Blog.

I have been writing about principles for a long time. Tenets. Calibration. Just right, over, and under. The 5Ps.

I recently built the BIQ and Porridge apps that use those ideas, and for a while each app carried its own copy of “the principles”, which were actually just Amazon’s Leadership Principles.

It worked until the second company showed up. A leader I respect deeply at Arm wanted the BIQ app to work for her hiring managers at Arm. She pointed me at Arm’s version of Amazon’s LPs: The 10x Mindset.

Amazon’s Ownership and Arm’s Own It are the same but different. How could BIQ ensure it served the right questions, and provided relevant examples without a full bespoke stack specifically for Arm? Same for Porridge and “just right, under, and over”. I also started noodling on what would happen if the tools worked for dozens of companies.

The fix: I pulled the model out of the apps and built LLM-based generators that are app-specific. I found the journey interesting and figured you might as well…

Behavior is the unit

Getting clear on the lexicon almost always helps. I have never been able to distinguish tenet from principle. They were 100% synonyms for me. No longer.

A tenet is a guiding principle for one endeavor. It takes a stand and settles the calls data cannot. A leadership principle is a tenet whose endeavor is an organization and whose subject is human behavior. That is the whole difference.

A principle is only worth having when it decomposes into behavior: something a person can observe, teach, practice, and live with the appropriate balance. Calibration makes that visible. Each behavior can be written in a real situation as under, just right, or over. The two ends are what make a principle teachable instead of inspirational.

Companies carve the same behavior into different names, so two sets rarely line up one to one. A facet is the slice that does. Dawn Aerospace’s Better Than Yesterday shares only the “better every day” slice of Amazon’s Insist on the Highest Standards.

The core owns the model. The apps own the experience.

That model lives in a public repo, kindel/principles. It holds the lexicon, the taxonomy, and the checks. It holds no user interface. Company is a parameter. dive-deep is three different principles if you forget which set you are in.

The sets today are Amazon’s Leadership Principles, Toyota’s The Toyota Way, Arm’s 10x Mindset, Coupang’s and Delivery Hero’s Leadership Principles, GitLab’s CREDIT values, and Dawn Aerospace’s Company Tenets. I have a backlog of a dozen more. A new set starts as an issue. The validator fails if the model is lying.

kindel.com mounts the apps. The pages you click are the experience. The model underneath is one set of files.

The Apps

I built these to accelerate the skills I teach. If we have worked together, you have already used some of them. If we have not, each one links the posts that explain the idea. They live at kindel.com/apps.

BIQ
There is an app for this. BIQ lets you pick a principle, get example questions, and compare hire and no-hire answers.

Porridge
There is an app for this. Porridge walks a principle as Just Right, Over, or Under.

Tenets
There is an app for this. Tenets takes the endeavor you type and opens an agent already briefed to write the stands.

5Ps
There is an app for this. 5Ps walks Purpose, Principles, Priorities, People, and Plan so an agent can write the first draft.

D × V × F > R
There is an app for this. D × V × F > R scores Dissatisfaction, Vision, First steps, and Resistance for a change you name.

SBI
Later. Situation, Behavior, Impact. Feedback about what happened, not who they are.

CBTO
Later. Customer, Business, Technology, Organization: a stack rank of the four lenses.

If you just want to use the tools, start at kindel.com/apps/.

As usual, I would be interested in hearing where this is wrong.

At the WeAreDevelopers World Congress 2026 in Berlin, I spoke with Luca Mancinelli (Solutions Engineer, Apryse), about a part of the AI stack that tends to receive far less attention than models themselves: the document pipeline.

As companies move more of their knowledge into AI applications, PDFs, scans, reports, and invoices are becoming key data sources. The real challenge is preserving enough structure and context for AI systems to understand them properly.

Document extraction is becoming key in AI applications

For much of the history of enterprise software, documents typically sat at the end of a process. An application generated a report, invoice, or contract, and the PDF was the finished product. Mancinelli believes AI is changing that relationship:

Documents used to be the output – for example, a PDF you’d print or send out. Now they’re also a structured input that powers AI systems.

More precisely, the challenge for developers is turning documents that are often unstructured or semi-structured into structured information that downstream systems can use.

An LLM can ingest raw text, but simply extracting characters from a file does not always preserve what makes the original document meaningful. A heading has a different role from a paragraph. A value in a table depends on its row and column. A number beside “Total” means something different from the same number appearing somewhere else on the page.

That is why document extraction is becoming an important preprocessing layer in AI applications. The goal is not just to recover text, but to give models cleaner, better-organized context.

Bad PDF parsing can lose information that even a stronger model cannot recover

Anyone who has tried to build a production pipeline around real-world PDFs quickly discovers that documents are far less predictable than API responses or database records. Mancinelli points to three recurring problems:

First are complex layouts: you can have headers, tables, and different formats in the same document, and that creates a huge challenge for systems. Second is poor OCR, which can produce messy output. Third is the loss of context.

This distinction becomes especially important with RAG systems and other LLM-powered applications. If a document is badly parsed before it reaches a model, a more powerful model cannot necessarily recover information that was lost earlier in the pipeline.

For developers, that means data quality becomes an architectural issue rather than simply an AI-model issue.

Document processing depends on the use case

Where document processing happens is another architectural decision that becomes more important as AI moves into regulated and data-sensitive environments.

Mancinelli argues against treating cloud and self-hosted processing as an either-or decision. The right model depends on the application, its infrastructure, and the type of data involved.

For organizations working with particularly sensitive documents, processing files within their own environment can reduce the amount of data that has to move outside their infrastructure. Cloud services, meanwhile, can lower the infrastructure burden and make experimentation easier in some projects.

It’s not one or the other; each solution has its own merit. It depends on the use case.

That may sound obvious, but it becomes a meaningful engineering trade-off once document pipelines start handling contracts, financial records, healthcare documents, or other sensitive enterprise information. Privacy requirements, latency, scalability, and operational complexity all have to be considered alongside extraction accuracy.

A powerful SDK is useless if developers struggle to integrate it

There is also a more familiar developer problem: integration. No matter how sophisticated a document-processing engine is, developers eventually have to put it into a real application, maintain it, and debug it.

For Mancinelli, good documentation is therefore one of the most important characteristics of a production-ready SDK. Practical examples matter as well: developers should be able to see realistic implementations and adapt them instead of building every integration from scratch.

He also points to higher-level APIs as a way of reducing unnecessary implementation complexity:

The point is particularly relevant as AI infrastructure becomes more fragmented. Developers already have to connect models, vector databases, data sources, evaluation systems, observability tools, and application logic. Adding another complicated layer to the stack can quickly increase both development and maintenance costs.

A useful document SDK therefore has to solve two problems at once: the difficult underlying document-processing problem and the developer-experience problem surrounding it.

Data pipelines will become increasingly important

Predicting what software development will look like several years from now is difficult, and Mancinelli avoids picking a single technology that will dominate.

Instead, he expects data pipelines to become increasingly important as companies process larger and more complex collections of information.

One thing is sure: the complexity of the data is going to increase, and the amount of data is going to increase as well. That means data management pipelines will become even more critical.

For developers working with AI, that may be the more practical takeaway.

“The industry tends to focus heavily on what happens inside the model: larger context windows, new reasoning capabilities, faster inference, and increasingly capable agents. But production AI systems also depend on everything that happens before a prompt reaches the model”, Mancinelli said.

If the input is incomplete, badly extracted, or stripped of its structure, even an advanced LLM starts from a disadvantage.

As enterprise AI matures, document processing may become less a peripheral PDF problem and more a core part of the AI data stack, one where developers have to think carefully about structure, privacy, deployment, and, above all, context.

The post AI Is Only as Good as the Document Pipeline Behind It appeared first on ShiftMag.