The scariest number here: 3.61% of CPUs in one large-scale study were found to cause silent data corruptions. Not “a few bad chips.” Nearly 4 out of every 100 processors doing math wrong, silently, with no error log. Google coined the term “mercurial cores” in 2021 after their production teams kept blaming software for data corruption. They’d debug for weeks, find nothing wrong with the code, swap the machine, problem gone. The actual cause: manufacturing defects at sub-7nm that pass every factory test, then degrade unpredictably months or years after deployment. Facebook confirmed the same thing independently. Hundreds of affected CPUs across hundreds of thousands of machines. The defect doesn’t crash your system. It just gives you 5 instead of 6 when you multiply 2x3, under specific microarchitectural conditions, with zero indication anything went wrong. Now think about what this means for AI training. A single corrupted GPU or CPU in a distributed training cluster doesn’t just produce one bad output. It feeds corrupted gradients into a synchronization step that gets averaged across every accelerator in the cluster. One bad chip can silently poison an entire training run. NVIDIA published a whitepaper on exactly this problem. Loss spikes during LLM training that nobody could explain traced back to silent hardware corruption. The part that keeps infrastructure engineers up at night: traditional defenses don’t work. ECC memory can’t catch this because the corruption happens during computation, not storage. Checksums like CRC heavily use vector operations, which are themselves one of the most vulnerable instruction types. The tools designed to detect corruption are running on the same flawed silicon. Google’s current detection method? Roughly half human-driven, half automated. And of the machines humans flag as suspicious, only about 50% are actually confirmed mercurial on deeper investigation. We’re debugging trillion-parameter models on hardware where we can’t reliably tell which chips are lying to us. Moore’s Law gave us more transistors. It also gave us transistors we can’t fully verify.

My least favorite coding practice is leading commas in SQL 🤢 "Oh it's easy to comment out a line-" It's easy to wear sweatpants out to dinner. But we live in a society

NEW: THREAD: A new ruling from Judge McBurney in Georgia overturning the abortion ban and allowing the procedure to continue has some REMARKABLE quotes. Let's take a look at just a few. 1/

We’d like to announce that while we are not a hotel, we are a concept of a hotel.