Inquiry into deadly Optus emergency call outage found failures in firewall upgrade

An inquiry into Optus's emergency services telephone number outage in Australia, which resulted in two deaths, revealed a series of failures during a firewall upgrade that left hundreds of people unable to contact police, fire and ambulance services.

Australia's second-largest telecommunications provider, owned by Singaporean firm Singtel STEL.SI, published the review on Thursday, which found at least 10 mistakes during the routine network upgrade carried out after midnight on September 18.

The independent internal investigation revealed that Optus provided incorrect instructions to its contractor, Nokia.

Additionally, the Finnish telecoms equipment maker selected and approved an incorrect method to proceed with the upgrade, which shut down key equipment but did not redirect calls, according to the report.

Nokia was not immediately available for comment.

The error impacted both normal voice calls and the emergency number Triple Zero, but the consequences were different, said Kerry Schott, the report's author and transport and infrastructure business executive.

While voice calls found an alternate pathway and their service continued, Triple Zero calls failed.

The outage lasted for almost 14 hours, during which 605 callers sought Triple Zero emergency services and assistance of some kind. About 75% of these calls could not connect, with two of them resulting in fatalities.

"Three issues are clear during this incident. The first is the very poor management and performance within networks and their contractor, Nokia," the report said.

"Process was not followed, and incorrect procedures were selected. Checks were inadequate, controls avoided, and alerts given insufficient attention."

The review found gaps in process, accountability, escalation, and information protocols, and highlighted "challenges in Optus' culture that have impacted decision-making and response times", Optus said in a statement.

The review made 21 recommendations, Optus said, including strengthening network change-management and escalation processes, improving incident detection and crisis response for emergency services outages, among others.

At its meeting on December 16, the Optus board accepted all the recommendations and "agreed to move swiftly with their implementation".

"The board is taking further action in relation to individual accountabilities flowing from the incident, which will extend from financial penalties through to termination in appropriate cases," Optus Chairman John Arthur said.