Troubled evacuation in Paradise: How a malfunctioning alert system kept citizens in harm’s way
A PBS documentary details the devastating Camp Fire in Paradise, California, and suggests the tragedy could have been much less catastrophic had potentially life-saving emergency alerts gone out as intended
The wildfire also could have been much less catastrophic, investigators for the film learned, had Butte County officials been able to successfully send emergency alert notifications.
November 8, 2018
“The morning the fire sparked,” explains a teaser for the film, “a single staff member at the Butte County Sheriff’s Office coordinated the evacuation orders, using a newly contracted software package called CodeRED.”
While this software allowed officials to send either zone-by-zone or mass alerts, there was a hitch: Zone-by-zone alerts, which the official chose in an effort to mitigate panic and keep evacuation routes moving, were set up as an optional service.
On the day of the fire, less than half of Paradise's 26,000 residents had registered to receive these notifications.
And this wasn't the only break down of critical emergency communication. As the fire worsened, the staffer again used the CodeRED system, this time to send mass alerts through the Integrated Public Alert and Warning System (IPAWS), the free FEMA resource that sends wireless emergency text alerts (WEA) to all mobile devices in a specified area (an opt-out service rather than an opt-in one).
But these messages never went through, and the county is still working with the software company to find out why.
Alerts Often Fail
This isn't the first time problematic emergency alerts have made headlines.
They did, however, use a different system that allowed them to tailor the outreach, much like the CodeRED system. But by the time these alerts were sent, 80 cellphone towers were badly damaged or knocked out entirely.
The previous year in Tennessee, Sevier County officials admitted that emergency alerts were not sent out at all ahead of a wildfire that spread through Gatlinburg, killing 13 people. The officials blamed a breakdown in communication between the city of Gatlinburg, the National Park Service and the Tennessee Emergency Management Agency "due to disabled phone, internet and electrical services."
And just this month, PG&E faced even more criticism over its preventative power outages because they also impacted cellphone towers, a critical lifeline for many residents in harm's way.
How Can We Make Emergency Alerts More Effective?
"Whether or not a blast of WEA messages would have improved the outcome in Paradise is almost impossible to know," said Art Botterell, a former emergency services coordinator with the California Governor's Office of Emergency Services.
Botterell resigned his post two years ago after growing "disappointed to the point of rage" over the state's attitude toward public alerts.
It’s generally better for people to know what’s going on than not," he continued. "The alert helps people confirm their speculation about what’s going on.”
The U.S. Department of Homeland Security Cybersecurity and Infrastructure Security Agency (CISA) agrees.
"Emergency alert, warning, and notification (AWN) systems help protect lives and property by identifying information about an impending threat, communicating that information to those who need it and facilitating the timely taking of protective actions," the agency explains in the executive summary to its recent report, Public Safety Communications: 10 Keys to Improve Emergency Alerts, Warnings & Notifications (access and download the full report below).
The 10 keys are as follows:
- Establish governance
- Identify and coordinate with others
- Develop plans, procedures and policies
- Account for diverse populations
- Maintain security and resiliency
- Incorporate safeguards
- Train, exercise and test systems
- Eliminate issuance and dissemination delays
- Deliver actionable messaging
- Monitor and correct misinformation
While each of these tips is essential to a robust public safety communications program, one is particularly relevant here: maintain system resiliency.
What Makes a Communications System Resilient?
As CISA again explains in a report, this time their Public Safety Communications Network Resiliency Self-Assessment Guidebook (which can also be accessed below):
Communications resiliency means a network can withstand damages, thereby minimizing the likelihood of a service outage."
But, the report also points out, "one of the most critical and vulnerable parts of these networks is often overlooked: the local access network. The local access network is the 'last mile' connection between an organization’s on-site communications infrastructure and the service provider’s network. In the event of an emergency, such as a cable cut, flood, or damage to the service provider’s facility, the local access network may be lost, leaving an organization unable to perform critical functions."
This is why they recommend a three-pronged approach to ensuring resiliency: route diversity, redundancy, and protective/restorative measures.
#1 Route Diversity "is defined as communications routing between two points over more than one physical path with no common points."
In the event that cell towers lose power or are damaged too greatly to function, agencies should have a backup communications route to make sure messages can still be delivered. While Butte County's CodeRED system did indeed provide an alternative messaging route via landline phones -- residents could register both landline and mobile phones to receive alerts -- the opt-in nature of the program combined with the fact that landlines are quickly becoming obsolete meant that the earlier zone alerts went mostly unheard:
Only 7,000 of the 52,000 people who evacuated received alerts about the wildfire.
These startlingly low numbers led the Butte County Grand Jury to conclude that "the CodeRED system’s dependency on telephone service is an inherent weakness of the warning system."
The Grand Jury report also noted that the CodeRED evacuation orders that did go out were likely disrupted by burning communication cables and cell towers, though it isn't clear at what point these routes became unusable.
#2 Redundancy means "that additional or duplicate communications assets share the load or provide back-up to the primary asset."
Because all communications leaving the Butte County Sheriff's Office were directed through the CodeRED system, this created a single point of failure, which unfortunately came to fruition when the system failed to connect to IPAWS -- a system that could have quickly disseminated a single message via a variety of channels rather than just phone service.
#3 Protective measures "decrease the likelihood that a threat will affect the network, while restorative measures, such as ECD’s priority telecommunication services, enable rapid restoration if services are lost or congested."
As a recent New York Times article explains, many cell towers do indeed have backup power sources to keep communication lines open in the case of an emergency, but there is no federal law in place to mandate any standards here.
And the result is a hodgepodge of solutions instituted at each carrier's discretion, with many towers running out of power almost immediately following electrical service outages.
Indeed, California's Public Utilities Commission has largely placed the responsibility of protecting these communication routes on the public. A recent report stated that it's “the responsibility of the customer to obtain the required backup power in the residence to have working telephone service during an outage event.”
Never mind the fact that, again, most of these households most likely do not have landlines.
Review CISA's 10 Keys to Improving Emergency Alerts, Warnings & Notifications:
Learn how to assess the resiliency of public safety communications networks: