Pro poker player here. I assess risk for a living in situations where outcomes are unknown. For example, when I call a bet from my opponent, I can't know for certain what two cards they're holding. The best I can do is think through all the possibilities, assess my risk and reward, then place my bets accordingly. I've refined this process over millions of hands of poker and 20+ years at the table. I use a simple to follow risk/reward framework, which can easily be applied to real world scenarios. For example, like whether to build superhuman AI (AGI) that could help us achieve immortality or end humanity. How do we decide whether it's worth the risk when we have no clue what the outcome will be? Well, there's a principle in poker called "asymmetry". It describes situations where the outcomes are unequal. For example, you enter a poker tournament for $10,000 where the first place prize is $10M. Worst case you lose $10K, best case you 1,000x your investment. It's a great risk/reward and a classic "asymmetric opportunity" that poker pros dream of. But the knife cuts both ways. There are plenty of situations where the asymmetry is to the downside. Lots of risk, low reward. Pros try to avoid those. The most important principle in bankroll management is that you never take a bet you can't afford to lose. Even if the potential upside is great, a pro never risks it all on one hand, one game, or one tournament, because there is simply too much variance, luck and factors beyond their control. Too many unknowns. At a certain point, protecting existing capital is more important than gaining additional capital, because if go broke you not only lose all your money, but all of your future earning potential too. Arguably the most important and under rated aspect of being a poker pro is managing risk well enough to simply stay in the game. As I learned the hard way after round tripping a million dollar bankroll before I turned 21, you build up slowly, but can go broke quickly. This bring us to AI, which is really just a classic hand of poker. There's massive upside - solving aging, curing all disease, advancing human knowledge, becoming interstellar, etc. There's also cataclysmic downside – like humans going extinct if this goes haywire. Thinking over both the risk and reward, it's clear the asymmetry of building superhuman AI is to the downside. My thesis is simple: no future gain in productivity is worth the risk of ending civilization. The bulls argue that "the probability is low", but that fails to acknowledge that not all risk is created equal. The reason it's not worth playing Russian roulette – even if the payoff is high – is because the potential gain isn't worth dying for. That's true even if the probability of dying is low. Accelerating civilization to superhuman AI while failing to solve the alignment problem is akin to playing Russian roulette with humanity. Naturally, this is dumb and should be avoided. We should continue progressing, of course, but only so long as we don't take on risks we can't afford. We can literally "go broke" (extinct) in one fell swoop if superhuman AI isn't architected carefully. One thing poker teaches you is that "bad beats" ( improbable hands that don't go your way) actually happen more often that you'd first expect. Sometimes there's only one card in the deck that can help your opponent, and sure enough, it comes at the end and you lose all of your chips. The only reason pros survive is because they manage risk and capital in such a way that they can continue playing the game. Superhuman AI, if misaligned, is like putting all your chips on the table and risking them in one hand of poker. It doesn't matter if the odds are favorable. If luck doesn't go our way, we lose everything. Once we're extinct, we can't play anymore. As I'm sure you'd agree, that's not a great outcome. We don't get another buy-in at the table, or a second shot at this. As we say in poker, we're all-in! I hope my poker lens helps clarify the situation in laymens terms. TLDR; Even though the probability of catastrophe due to AGI may be low, we must first defend against unaffordable risks, and make progress second. This is arguably the most pressing issue of our time. If this resonates, please share for awarness. If anyone working in AI at @AnthropicAI, @OpenAI or elsewhere would like to discuss further, I'm happy to help in any way I can - feel free to DM. As we say in poker when the stakes are high and we're praying to win, "onetime!"