Skip to content
InfoQ·

🚨Amazon Outage Shows Progressive Collapse in Digital Systems

An AWS error took down half of the internet last year

TL;DR

A small DNS update error caused a major outage affecting services like Alexa, Slack, and Shopify. This highlights how digital systems can suffer progressive collapse similar to structural collapses.

An AWS error in DynamoDB led to the deletion of DNS routes in Route 53, causing widespread outages across multiple services including Amazon's own Alexa and external platforms like Slack and Shopify. The incident underscores the concept of progressive collapse, where a minor failure can trigger a cascade effect impacting larger systems. This is critical for developers managing distributed systems as it highlights vulnerabilities that could lead to catastrophic failures if not addressed. For instance, in October 2021, a faulty nut triggered a gas explosion at Ronan Point tower block, leading to the collapse of four floors due to load-bearing wall failure. Similarly, digital systems can suffer from similar cascading effects when small errors go unchecked.

Amazon Outage Shows Progressive Collapse in Digital Systems — InfoQ

Key Points

1

A faulty DynamoDB update caused Route 53 DNS routes deletion, leading to a major outage on October 20, 2021

2

The Ronan Point tower collapse was triggered by a gas explosion from a faulty nut, causing four floors to fall in 1968

3

Network load balancers, compute services, queues, and EKS were affected during the AWS outage, impacting customer-facing products

4

Amazon's Alexa stopped working alongside Slack, Snapchat, Zoom, and Shopify due to the DNS route deletions

5

The concept of progressive collapse from civil engineering applies to digital systems, highlighting potential vulnerabilities

Why It Matters

If you're managing distributed cloud services, this AWS outage shows how a small error can cascade into major failures. For instance, DynamoDB's update mistake led to Route 53 DNS route deletions, impacting network load balancers and compute services. This highlights the need for robust fail-safes in digital systems.

awsdynamodbroute-53outageprogressive-collapse

Frequently Asked Questions

Why does this matter?

If you're managing distributed cloud services, this AWS outage shows how a small error can cascade into major failures. For instance, DynamoDB's update mistake led to Route 53 DNS route deletions, impacting network load balancers and compute services. This highlights the need for robust fail-safes in digital systems.

What happened?

A small DNS update error caused a major outage affecting services like Alexa, Slack, and Shopify. This highlights how digital systems can suffer progressive collapse similar to structural collapses.

Comments

Subscribe to join the conversation...

Be the first to comment

Enjoyed this article?

Get it daily. 7am. Free. Reads in 5 minutes.

Join 3,211 builders reading daily.

Also get