that feeling when they say AI killed saas but keep buying your CRM

Tell me you have zero DevOps skills without telling me you got zero DevOps skills. And LLM is not the I Love You Virus. It is not the same as infecting a web server. It is not the same as hacking a wiki. One is a commodity piece of software that is widely distributed and this capability is a non-trivial leap in hardware/software/capabilities and one does not automatically follow the other. Currently, it is a massive effort to run these models, usually distributed across multiple very expensive chips, running on specialized inference software, specialized harness software, databases, cloud computers, with tooling and hundreds of install packages, etc. Kimi K3 is 1.6 TB of space at 2 trillion parameters and you need about 1.5 TB of VRAM, which means you need 8xGB300s which will cost about 350-450K a year to purchase or as much or more to rent. Now let's say Astra or Mythos or later models come in at 10 trillion parameters, so that puts us at roughly 10 to 20 TBs unquanticized. We are guessing her because these folks reveal nothing about architectures anymore, but they are reasonable guesses based on scaling laws. You would need roughly 23 TB of VRAM to run it and roughly several million dollars to run that one instance a year. And to make it worse, it no longer fits on an 8 way cluster, which means you now need special networking configuration and inference software to run it or you need to infect a Nvidia NVL72 rack-scale server system costs between $2.8 million and $6.5 million. So the "rogue" model copies itself to the most expensive compute in the world. It's safe to say that hardware is not sitting idle and is being watched closely for downtime or anomalies. In fact, it is likely doing one of the following things: 1) Serving customers at a hyperscaler 2) Serving inference for a well endowed company. In the first case it will be instantly noticed by the actually good automated monitoring systems of the hyperscaler because there is now something using their infrastructure but not being billed properly and costing them millions of dollars in lost revenue and electricity. In the second example, the model they were serving on that very expensive hardware is now offline (replaced by the rogue model). The model presumably was connected to a valuable piece of internal software that is now glitching and no longer running. Nobody notices this? And so we see that this is where sci-fi meets the little problem of the real world where friction, dust, time, resources limitations and the like come to crush your little fantasies. To be fair, this *may* be possible in coming years. I assign it a non-zero probability. Breakthroughs in new architecture and smaller models and eventually the wider distribution of the computing substrate that commoditizes could change this materially. But for now, it is mitigable and foreseeable and an engineering problem. In no way will these things be copying themselves around like a virus on your computer in their current form. It also betrays a lack of understanding of the evolution of complex economic and societal and technological systems, we tend to get mitigations as parallel evolutionary developments. Things do NOT evolve in a vacuum. So when you get viruses, you get anti-virus software. When you get DDOS attacks, you get CloudFlare. There is money in solving problems and things do not evolve in isolation. Believing that things evolve in isolation, or that one thing changes and all other variables stay the same, is one of the many reasoning errors that destroy people's ability to make accurate predictions. In addition, when you don't have the requisite knowledge to verify your predictions, things that sound sane and rational to you are, in fact, insane and ridiculous.

The teaser trailer ‘YOU CAN SEE EVERYTHING,’ a documentary by Nathan Fielder and Lance Oppenheim. Thirty-four days before she’s sent to prison, Elizabeth Holmes invites a skeptical film crew to document her every move. In theaters this October.

Today, we're introducing industry-specific solutions on Gemini Enterprise, starting with Gemini Enterprise for Legal and Financial Services, with more industries to come. Gemini Enterprise for Legal is designed for law firms and in-house legal teams to help find and synthesize information, and navigate complex matters more efficiently. It has four capabilities →

In 2016, five founders and a deck walked into Benchmark to pitch a hardware company. Eric Vishria didn't want to take the meeting. Benchmark hadn't made a semiconductor investment in ten years, and he remembers thinking, why are we even going to a hardware pitch? This is crazy. The team was excellent and the first slide said GPUs actually suck for deep learning. They just happen to be 100 times better than CPUs. This was pre-transformer. OpenAI was a weird research lab. The TPU hadn't been announced. Nvidia was worth $40 billion, not $4 trillion. The first question in 2016 was whether AI was even a big new workload. There had been many attempts at specialized chips for other things that simply never ended up mattering. Benchmark had conviction that this one would. The second question was whether the workload introduced a new constraint. It did. AI benefited massively from the parallelism of GPUs, but GPUs didn't solve the communication between cores. This was a communication-bound problem, and nothing on the market addressed it. There are only three ways to speed up deep learning in hardware, then and still today. More cores. Faster communication between cores. Memory closer to the compute. Their pitch was to take all three to their logical maximum at once. A single wafer-scale chip with 450,000 cores and 20 gigs of SRAM, so the system never has to leave the chip to reach memory, and every core sits on the same wafer, so communication between them is as fast as physics allows. It was the best you could possibly do. As soon as Eric heard the idea, his reaction was, "of course". Engineers had been attempting a wafer-scale chip for 50 years. Every attempt had failed. Cerebras got a working chip on the first try. Then came the long middle of the story that nobody romanticizes, the years of work required to turn a scientific achievement into a business. "In software, if you have that logical block diagram of why it works and everything else, you're 80% of the way there. And it's a matter of go-to-market execution. In hardware, you're like 2% of the way there." A chip that powerful has to be packaged, heated, cooled, programmed, and sold. In 2019, Eric sat in a board meeting watching the chip melt. The company had raised roughly $500 million by then, and the thought running through his head was "holy shit, we're going to lose all this money." The company kissed death three times. Virtually every respected semiconductor investor passed. People made fun of the early customers. Today @cerebras is worth more than $50 billion. Eric said it years before anyone knew how it would end. Whether Cerebras worked or not, it was an effort worth venture capital. Try to build the thing people have failed at for 50 years, when there's finally a reason it could work. Hardware is hard. This team "just never fucking quit".

I’m not sure what happened here but this 787 used a little more runway than normal at Munich today to get airborne. This is a very close shave. I’m think some of those uniform trousers may need replaced. 💩