Contents:
The popular social media platforms Facebook, Instagram, WhatsApp, and Messenger, are back online following a massive disruption that lasted almost six hours. Billions of people worldwide have been unable to access the platforms because of a BGP routing issue.
Yesterday evening, all the platforms owned by Facebook crashed, with browsers showing DNS errors when users were trying to open them.
As showed by BleepingComputer, when trying to connect directly to the Facebook DNS servers displayed below, they were also inaccessible.
Disruption Caused by a BGP Routing Problem
While the problem initially seemed to be DNS-related, it was eventually discovered that the situation was much worse than that.
Giorgio Bonfiglio, a Principal TAM at Amazon AWS, explained that several Facebook routing prefixes had mysteriously vanished from the Internet’s BGP routing tables, effectively rendering any services hosted on their IP addresses inaccessible.
A bunch of Facebook networks has just disappeared from the internet: pic.twitter.com/j07LrmAAdW
— Giorgio Bonfiglio (@g_bonfiglio) October 4, 2021
According to Wikipedia,
Border Gateway Protocol (BGP) is a standardized exterior gateway protocol designed to exchange routing and reachability information among autonomous systems (AS) on the Internet. BGP is classified as a path-vector routing protocol, and it makes routing decisions based on paths, network policies, or rule-sets configured by a network administrator.
A network must advertise its routes, or prefixes, to the rest of the world to be visible on the Internet. No one else on the Internet will be able to connect to their servers if those prefixes are removed.
As the social media platform set the company to use a domain registrar and DNS servers hosted on their own routing prefix, no one could access the IP addresses or the services running on them when those prefixes were deleted.
The Facebook routing prefixes started to be visible again on the BGP routing table at other networks. Users were available to connect to Facebook, Instagram, and WhatsApp once the prefixes were visible on the Internet.
Can see 122 routes now, something is coming back online!
DNS servers 185.89.218.12 and 185.89.219.12 are back on the internet, at least. pic.twitter.com/EflBjctemw
— Giorgio Bonfiglio (@g_bonfiglio) October 4, 2021
The California-based tech giant declared that it had “no evidence that user data was compromised as a result of this downtime.”
So far, Facebook hasn’t said much about what went wrong or how the problem was fixed, although it had been reported that the company dispatched a technical team to its California servers to manually reset the servers where the issue originated.