A podcast about why things break on the Internet.
Joined March 2021
- Tweets28
- Following9
- Followers287
- Likes11
OK, if Facebook publishes an RCA that contains the term "angle grinder" we will absolutely do a special episode on the outage.
We just wrapped up season one of the podcast with the most cliched episode title we could think of: downtimeproject.com/podcast/… We'll be back with season 2 soon -- send us ideas for outages you want us to cover!
For episode 10, we took a look at AWS’s Kinesis outage from November 2020. You know you’ve truly reached web-scale when the number of servers in your cluster exceeds the OS thread-count limit: downtimeproject.com/podcast/…
During this episode we also mentioned one of @tklein's favorite historical outages: S3’s 2008 outage which combined single-bit corruption with gossip protocols. If you haven’t read it, definitely check it out: status.aws.amazon.com/s3-200…
This week we discuss @coinbase’s thundering herd outage from November, 2020. Listen as Tom and Jamie go through a refreshingly non-database related outage and stretch several animal metaphors to the breaking point: downtimeproject.com/podcast/…
This week we discussed @auth0status’s 4 hour, 20 minute outage from just a few weeks ago: downtimeproject.com/podcast/…. Check out the episode to hear some thoughts on cache design, engineering leadership, and keeping your databases happy.
The Downtime Project retweeted
Really interesting hearing a third party analyse your incident based on a public post-mortem.
It's a good analysis too; pragmatic, avoids hindsight bias and counterfactuals, and puts equal focus on the positive capacities.
We just published our third episode, where we talk about @monzo's outage in 2019 when adding servers to a Cassandra cluster: podcasts.apple.com/us/podcas…
Episode 6, "GitHub’s 43 Second Network Partition" is now out! We go through @jasoncwarner's write up of @github's 2018 outage and spend a great deal of time talking about all the fun you can have with MySQL replication: downtimeproject.com/podcast/…
Wow, what timing. We just released episode 5 yesterday, which goes through the details of the 2018 Auth0 outage: downtimeproject.com/podcast/…
[status] Investigating: We are currently experiencing an increased error rate in Auth0 services. Our Engineering team is investigating, and we will provide you with further updates as we have them. stspg.io/n1xpdg0yhrk8?u=v0zz…
Episode 5 is out, talking about @auth0's 2018 outage: downtimeproject.com/podcast/… This includes some speculation about how something like @honeycombio might have been helpful -- @mipsytipsy did we get it right?
Episode 4 is out! This week we cover once of the best written and most educational postmortems we've seen: @jgrahamc's write up of the 2019 @Cloudflare regex issue. downtimeproject.com/podcast/…
And our second episode was about @gitlab's rough day in 2017 after a @PostgreSQL replication mishap: downtimeproject.com/podcast/…
In case you missed the earlier episodes, our very first episode talked about @SlackHQ's outage on the first working day of 2021. Our site has transcripts and links to the other episodes: downtimeproject.com/
We just published our third episode, where we talk about @monzo's outage in 2019 when adding servers to a Cassandra cluster: podcasts.apple.com/us/podcas…