That defect affected computers that use Microsoft’s Windows, which powers hundreds of millions of personal computers and many back-end systems for airlines, digital payment, emergency services call centers and much more.
The problem wasn’t a cyberattack or a security incident, CrowdStrike said.
GET CAUGHT UP
Stories to keep you informed
Computer network outages aren’t unusual, but experts said they were stunned that one company’s software error rippled through so many computer systems. They said it was a double whammy: An error in CrowdStrike’s widely used software took down computers powered by the dominant Windows software.
“We haven’t seen a cascading failure like this — maybe ever,” said Chuck Herrin, an executive with the digital security firm F5 Inc.
Marie Vasek, an assistant professor at University College London’s computer science department, said the widespread computer meltdowns showed how reliant global technology systems are on a small number of companies’ software, including that of Microsoft and CrowdStrike.
“The issue here is that Microsoft is a standard bit of software that everybody uses, and the bug in CrowdStrike is deployed to every single system,” she said.
Vasek said technology networks have become so sprawling, complex and interrelated that it increases the odds of one botched line of software code bringing down entire computer networks.
She and other information technology experts also said that because CrowdStrike’s digital protections are considered essential, its technology is given priority access on many computer systems. If something goes wrong with CrowdStrike software, that privileged access can grind computers to a halt.
In a statement, CrowdStrike said it is “working with all impacted customers to ensure that systems are back up and they can deliver the services their customers are counting on.”
Some companies affected by the CrowdStrike glitch, including banks and emergency services centers, said Friday that they had implemented CrowdStrike’s repaired software and were starting to recover from their computer network meltdowns.
Herrin said one challenge in recovery is that technicians might need to access back-end computer servers in faraway data centers to install a software update and restart the machines. He cautioned that flights, surgeries and payroll systems might have lingering problems in coming days.
Vasek said both Microsoft and CrowdStrike need to examine their procedures to prevent a repeat of such widespread technology failures.
She said CrowdStrike needed to consider how to safely update its software to many millions of computer networks. And Microsoft, she said, needed to do more to ensure that updates to software from other companies don’t cripple Windows machines.
“Microsoft needs to think about how to check that software is as it should be,” she said.
Microsoft didn’t directly address that criticism but said in a statement that the company is “actively supporting customers to assist in their recovery.”
The company had also reported outages with some of its popular web-connected software for corporate and government technology networks.
It wasn’t immediately clear how many of Friday’s computer network collapses resulted from the defective CrowdStrike software update and which were the result of problems that started Thursday with Microsoft online services and its corporate cloud computing service, Azure.
A spokesman for Microsoft said the company didn’t believe the CrowdStrike software bug was related to the outage that impacted a “subset of Azure customers.” It has been resolved, he said.
This is a breaking news story and will be updated.
