Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
165506 stories
·
33 followers

After the Outlook outage: Mary Jo Foley on Ignite, Microsoft’s AI pitch, and a new approach to the press

1 Share

This week on the GeekWire Podcast, longtime Microsoft beat reporter Mary Jo Foley, a GeekWire contributing writer, sizes up where the company stands after one of its most eventful weeks in years.

Microsoft opened its Ignite conference with an apology for the global Outlook outage, then took direct aim at OpenAI and Anthropic on price, even though it invests in both and sells their models. We discuss the lasting damage from the outage, why enterprise customers have few places to go even when they’re fed up, and Microsoft’s pitch to make Copilot the “OS for work,” as Anthropic goes public.

Mary Jo also explains how Microsoft changed the way it rolls out news at its big conferences, what got lost along the way, and what that says about the company after the shakeup of its communications team. Plus, we weigh who from Microsoft could lead Nvidia, and the GeekWire trivia challenge goes back to 1997.

Subscribe to GeekWire in Apple Podcasts, Spotify, or wherever you listen.

Related links and stories

From Mary Jo Foley on GeekWire

Edited by Curt Milton. Music by Daniel L.K. Caldwell.

Read the whole story
alvinashcraft
15 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

The FBI and USDA were reportedly hacked

1 Share
Digital photo illustration of a laptop displaying a black and green image of the Capitol building, with ones and zeroes falling in the background.

Friday, it was reported that both the US Department of Agriculture (USDA) and the FBI were hacked. In reporting on the FBI hack, CNN noted that this is the second such attack on the agency in the last few months, this time targeting the National Crime Information Center (NCIC), which federal, state, and local law enforcement use to share information, including warrants and missing persons. President Donald Trump later confirmed the hack in an interview with Fox News. NBC first reported the USDA hack on Friday. According to Reuters:

While the payments ​database is the known extent ​of the intrusion so far, the former ‌officials ⁠warned that …

Read the full story at The Verge.

Read the whole story
alvinashcraft
15 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Microsoft’s final Outlook outage report details testing gaps and safeguards that backfired

1 Share
Satya Nadella speaks onstage at Microsoft Ignite 2026 in front of the Microsoft logo
Microsoft Chairman and CEO Satya Nadella at Microsoft Ignite 2026 in San Francisco on Tuesday: “I know we have work to do to provide that resilience that you can count on.” (Photo by Dan DeLong for Microsoft)

The bug behind last week’s global Microsoft 365 outage struck only newer, more powerful servers in a key part of Microsoft’s network. That’s because it created a kind of traffic jam inside each server, and the more work a machine tried to do at once, the worse the jam got. The newer servers are built to do more at once, so they’re the ones that seized up.

One reason Microsoft didn’t catch it in advance: it was testing on older servers.

“Our testing did not adequately represent the combination of hardware and traffic mix encountered during this incident,” the company acknowledged in its final report, released Friday evening, a week after resolving the outage that crippled Outlook and cut off email for people around the world.

Testing that more closely replicates the company’s production environment is one of 13 next steps in the report — measures the company says it will take to keep it from happening again. The list is also aimed at reassuring the businesses that rely on Microsoft for email and collaboration, after CEO Satya Nadella acknowledged at the company’s Ignite conference this week that it has work to do on resilience.

Many of the other steps identified in the report go after a bigger problem: during the incident, the safeguards meant to contain the failure spread it around the world instead.

What set it off: As detailed in Microsoft’s preliminary report last weekend, the bug at the center of the outage had been sitting dormant since July, when Microsoft started rolling out the software update that introduced it. For more than three months, it caused no problems.

The bug affected the servers that make up what Microsoft calls its network gateway, a front door that connects people to Outlook email and other Microsoft 365 services. The flaw came from a Windows update used only on those servers. Standard versions of Windows weren’t affected.

So what was the bug? Microsoft’s final report gives a partial answer, saying it came from a change to how the gateway servers handle incoming connections.

Under heavy load, it turned out, the Windows process that sets up those connections bogged down, with tasks stuck waiting on each other until the servers ran out of computing power and memory and stopped responding.

The final report doesn’t say what the July update was meant to fix. But it does lay out three conditions that finally set the bug off. Without the bug, even all three together would have been routine, the report says.

First, nearly 14% of the gateway’s server capacity was already offline that day for routine software updates. That’s normally not a problem, because Microsoft keeps enough spare capacity to absorb it. But it meant fewer servers were available to carry the load.

That’s a different explanation from the preliminary report, which blamed the missing capacity on an unrelated problem.

Second, all of Microsoft Teams’ traffic landed on those same servers. Teams normally runs through a separate gateway of its own. About 15 hours before the outage, a routing glitch on that Teams gateway in Microsoft’s US East region set off an automatic switch that moved all Teams traffic worldwide onto the servers that handle email.

Third, the U.S. workday got going. Widespread failures began around 8 a.m. Pacific, as traffic in North America climbed toward its daily peak.

How it spread: As servers stopped responding, Microsoft’s systems automatically moved their traffic to healthy servers nearby, which had the same bug and buckled under the extra load. At the low point, more than 80% of the gateway’s servers in North America were unresponsive.

The system then shifted traffic to other regions, spreading the problem further.

“Within about two hours of the broad failures that began in North America, servers in every region were affected globally,” the report says.

The preliminary report had blamed that spread partly on “a separate issue.” The final report doesn’t repeat or explain that phrase. Instead, its findings say the way Microsoft distributes traffic “can cascade failures to other regions.”

Once the servers went down, getting them back up was its own problem. As servers came back online, they were swamped by pent-up demand and froze again, a pattern engineers call a “thundering herd.” Microsoft says it will ease traffic back onto recovering servers gradually. That fix doesn’t have a date yet.

The fixes: To keep a failure from spreading again, Microsoft says it will change how its systems shift traffic between regions, so a problem in one part of the world stays there.

It also plans to split up the servers that handle email, so a surge from another service like Teams can’t take them down. And Teams will switch over only in the region that has the problem, not worldwide.

Some of the fixes are due by the end of the year, others by the end of March. Several don’t have a date yet.

Microsoft says its first notice to customers went out about 20 minutes after the outage took hold, and it plans to shorten that by the end of the year. No business email was lost, according to the report, though some delayed messages to consumer mailboxes that had been inactive for more than 30 days bounced back to senders.

The report also confirms what Microsoft said the day after the outage began: it wasn’t hackers, and it wasn’t AI. “This incident was confirmed not to be caused by a cyberattack, and we found no evidence of malicious activity,” it says. “This change was not caused by AI tooling.”

The new testing is supposed to be in place by the end of March, according to the final report. This time, the company says, the machines in the lab will actually match the ones running the service in production.

Read the whole story
alvinashcraft
15 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Mozilla unveils new Firefox add-ons developer hub

1 Share
Mozilla has announced a completely redesigned and revamped Firefox add-ons developer hub. While this is certainly good news for developers of add-ons for Firefox, the fact that it makes life easier for these developers means that it is also very good news for users of the web browser. Fast facts: The Firefox add-ons developer hub has been around for 16 years now, and developers have long been of the mind that it was in need of an overhaul. Now Mozilla has delivered. The organization says: Today, we’re thrilled to announce a completely reimagined Firefox add-ons developer hub, rolling out this… [Continue Reading]
Read the whole story
alvinashcraft
16 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Random.Code() - Handling Private Members and Other Issues in SaferAccessors

1 Share
From: Jason Bock
Duration: 1:06:20
Views: 38

In this stream, I'm going to see if my little hack idea will work so I can consistently get non-public members in my source generator.

https://github.com/JasonBock/SaferAccessors/issues/10

#dotnet #csharp

Read the whole story
alvinashcraft
17 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

If we stop reading the code, what do we read?

1 Share
Motion-blurred stacks of glowing red, orange, and blue blocks, evoking the speed of AI agent code review

“Human beings are too slow when dealing with code. So if we’re going to move at the speed of the agents, we’re all going to have to back off a little bit and maintain the high-level supervision, but at the level of code, that’s probably out of our domain now.”

That’s Robert “Uncle Bob” Martin, who wrote Clean Code almost 20 years ago, and a whole generation of engineers learned from it that code is written for other humans to read. When I had him on The Hangar DX podcast, he told me he never sees 95% of the code his agents write.

I didn’t expect to hear that from him. I also don’t think he’s wrong about the speed. Agents open pull requests faster than anyone can read them, and most teams stopped reading line by line a while ago.

Next, he said, “I try to get as much intelligence about what it has done as I can without forcing myself to go all the way to code.”

“Even the person who just declared code out of scope for humans is still reading something.”

Even the person who just declared code out of scope for humans is still reading something.

He showed me what that something is. Partway through our conversation, he shared his screen and walked me through the tool he reviews with now.

An agent breaks his project down into something close to UML, and he can expand, collapse, and drill into any part.

Problems show up in red:

  • Functions with a CRAP score that’s too high. CRAP, short for Change Risk Anti-Patterns, is a 2007 metric that combines cyclomatic complexity with test coverage and spikes when code is both complex and poorly tested.
  • A mutation-testing score that’s too low.
  • Arrows marking dependencies that point the wrong way.

The agent is attached to the diagram, so he can run what-if scenarios, ask it to move components into different layers, and review the proposed structure before any of it becomes source code. He can still drill down to a module, but usually he doesn’t.

He’s checking the partitioning, the direction of the dependencies, and whether the parts that change often are decoupled from the parts that don’t.

“This was the old dream of UML,” he told me, “but UML was conceived back when it was a static thing that you did before you wrote the code. This is now a dynamic thing that you do after the code is produced.”

That works for one engineer and one agent. On a team, where other people need to understand the change too, what should the reviewer be looking at when they approve it?

Nobody reads anything

Ray Myers has spent a good part of this year arguing against the idea that we should stop reading code. When he came on the podcast, he summed up the people on the other side better than they do themselves: “Most of the people who are telling you don’t read code are actually doing some lowercase form of actually reading the code.”

Lowercase reading takes a few forms. Some engineers skip the code on throwaway, low-stakes tasks and read carefully everywhere else. Some lean on automated verification, so a passing test suite, a harness, or a dashboard becomes the thing they read. Some read selectively, skip the bug hunt, and look only at design and maintainability. Some read at the very end. None of them read nothing.

What to read when you stop reading the code

I’ve argued before that code review was doing two jobs at once, alignment and standards checking, and that AI broke each of them differently. The review I’d build today hands the reviewer three things to read.

Intent

What the change is for, what’s in scope, and what’s deliberately left out. Intent gets captured where decisions actually happen, in the prompts and the back-and-forth with the agent.

Acceptance criteria

These come in two layers. The first is the criteria for this specific change; the second is the invariants from the team’s AI slop registry, the patterns your senior engineers keep correcting, which apply to every matching change automatically. A verifying agent with no context from the agent that wrote the code checks each criterion and returns a verdict and evidence. This is how we built Verify at Aviator. The reviewer reads the criteria and the evidence with one question in mind: are we solving the right problem with the right constraints?

Decisions

The architectural, behavioral, and technical calls made along the way. Which library, which data model, what happens when the downstream service times out, what we chose not to build. Most of what an agent writes is boilerplate.

The decisions are what a human actually contributed, so another human should look at them. When two engineers disagree about one of them, the team needs a place to argue it out, and that deserves its own article.

Uncle Bob described the new human job as taking responsibility “just above the code level.” Intent, criteria, and decisions live there.

Keeping the mental model

Knowledge sharing and building mental models of the codebase were never side effects of code review; they were part of the job. Mitchell Hashimoto calls his version of this the whiteboard defense. Anyone who ships a customer-facing system should be able to explain how it works and defend the decisions behind it. He doesn’t expect them to remember function names. He does expect a confident answer when he asks, “Why did you do X instead of Y?” or “Where does this fail?”

Reading intent, acceptance criteria, and decisions keeps the mental model alive even if engineers can’t realistically read every line of AI-generated code. You still know why the system works the way it does and how its parts fit together, even if you haven’t looked at every function. And the team no longer depends on the one senior engineer who remembers how everything works.

Not reading the code is a sliding scale

Few people are closer to what code review looks like across the industry than David Poll, who leads GitHub’s pull request and code review team. When we talked, he told me that asking whether to read the code or not is the wrong question.

“The question is not whether I read or not read the code; it’s whether my attention gets directed to where it’s actually useful. Can I be pointed at the parts that are meaningful for me to go and look at and apply my judgment and expertise, and the things that can be with high confidence removed from my attention set, can fall away more and more over time.”

He’s right. Most teams already read some parts of their codebase closely and barely look at others. What lets you move along it safely is the quality of what you read in place of the code. Clear intent, criteria backed by real evidence, and decisions recorded where they were made let more of the code fall out of your attention set with confidence.

Someone still owns it

“The agents are not responsible for anything. They’re entirely irresponsible. Only humans can be responsible.”

That’s Uncle Bob again, and it’s the one part of this debate nobody can automate away. When I laid out the new shape of code review, I split it into five layers: argue, capture, codify, debate, and own. Own is the layer he’s talking about that no tool takes off your plate. The engineer who merges a change owns it, whether they read every line or none.

“The agents are not responsible for anything. They’re entirely irresponsible. Only humans can be responsible.”

You can stop reading the code. You can’t stop owning it.

The post If we stop reading the code, what do we read? appeared first on The New Stack.

Read the whole story
alvinashcraft
17 minutes ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories