Mehryar retweeted
Our #ApacheKafka streaming infrastructure served 29 million messages per second at peak. That’s 45% growth from 2022 – and we did it more efficiently this year with 14% fewer brokers.
🚀🚀🚀🚀🚀🚀
That's a wrap on another incredible BFCM! You got a nerdy preview from @tobi, and we're back with the final stats. Let’s go!🧵👇
We achieved 99.999+% uptime, handling 29.7 PB of data served from across our infrastructure over the entire event! That’s over 5 TB/min.
2
432
Very proud of the team preparing our Kafka infrastructure to handle record breaking traffic! 🚀
📈Our #ApacheKafka streaming infrastructure served 20 Million messages per second at peak, that’s a 52% growth from 2021!
1
9
Mehryar retweeted
Another Black Friday Cyber Monday (BFCM) weekend done and dusted!
We achieved 99.999+% uptime while averaging 3 Terabytes per minute of egress traffic across our infrastructure.
That’s 4.3 Petabytes per day! 🤯
Check out the thread for more awesome performance stats 🧵👇
Mehryar retweeted
The recording for my recent talk at #current22 on #Kubernetes StatefulSet migrations is now available: confluent.io/events/current-…
Mehryar retweeted
Enjoyed talking about how @ShopifyEngineering's #ApacheKafka team upgrades #Kubernetes at #Current22!
Slides: slideshare.net/VanessaVuiber…
👋🏼 The streaming platform team @ShopifyEng had an absolute blast participating in #Current22 the past couple of days! With the event wrapped up, here are some of our talks that you should definitely check out when the recordings are available 👇🏼🧵:
Lastly, if you're interested in learning about a data engineering use case at @ShopifyEng scale. Check out Tim's talk:"The Metamorphosis of Database Changes - How We Turned CDC Streams into Iceberg Datasets". 🧊
1
7
For more information and the abstracts check out: 2022.currentevent.io/website… #Current22
Mehryar retweeted
@ShashPand and I just published an article on Data Discovery & Lineage for an Event Streaming Platform link.medium.com/ezhYGFcq6qb Please have a read and let us know your thoughts. #datagovernance #datalineage #eventdrivenarchitecture
Mehryar retweeted
Our whitelist sale will be on 23rd April at 7pm EST. To get on the whitelist you must follow @TheGivingNFTs and complete this form to provide your wallet address: forms.gle/co1dHJkeiKGQohYy6. We could really use some help getting the word out so please retweet 💕 #NFTWhitelist
Congratulations to the @redpandadata team! They are doing fantastic work in the streaming space that I encourage everyone (especially my fellow Kafka peeps) to check out! 🎉🎉
This quoted post is unavailable.
Having been part of a team that’s been building a platform that’s grown significantly over the past couple of years, this 🧵 is spot on.
This is the most common question I'm answering in recent fireside chats...
Q. How do you handle/manage the stress of working in a hyper-growth company where there's always too much to do?"
A. It's a mindset change: problems upgrade but don't go away. Let me explain... 🧵
1