Software engineer & independent researcher building AgentFarm: evolutionary agents, emergent agency & open-ended dynamics. Overachieving midwit.
Arizona
Joined June 2012
- Tweets1.7K
- Following209
- Followers81
- Likes3.9K
Pinned Tweet
Silicone valley VCs (i.e. YC) are like prosperity churches to me. The more money you produce for the church, the more moral/saved you are. Hence, you must produce more money. Not a perfect comparison obviously.
Founders culture is now a kind of superficial caricature of the American spirit.
Chris Mangum retweeted
YouTube plans to introduce AB testing for videos. I agree with many others that this is a bad idea.
Maybe you squeeze out some more watch time in the short term, but undermining a sense of a shared viewing experience among the audience is too steep a price.
Chris Mangum retweeted
The arrogance that can come disguised as altruism is completely selfish. That’s the paradox: you get to claim goodness and moral superiority without personally experiencing the consequences of what you advocate for. Once you do, theory rarely holds up.
Woke NYT, 1619 Project writer deliberately sent daughter to troubled schools as a 'sacrifice' - and it backfired spectacularly trib.al/68LXBWJ
Chris Mangum retweeted
In the age of AI, I think you should dramatically increase your level of ambition
Chris Mangum retweeted
I went to an rationalist/EA AI doomer event last night with the goal of meeting someone who could calmly explain to me the argument for how they get to the conclusion that AI will kill everyone.
Because these are rationalists, I expected to find people who could lay out a serious extinction pathway, walk through the assumptions and intermediate steps, identify choke points where negative pathways could potentially be regulated, and entertain disagreement about where their assumptions might break down.
My experience was the exact opposite.
The people I spoke to seemed psychologically unable to entertain the possibility that their conclusion (that AI literally kills everyone, which was a point that many would not even budge on) might be wrong.
Their most extreme conclusion was treated as an axiom.
I had multiple people walk out of conversations with me when I questioned their inevitable extinction premise.
At one point during a panel, the moderator asked everyone in the audience who was still skeptical of AI doom to raise their hands. About half the room did. He then told all of the skeptics to leave the panel and go with one of the speakers, who would answer their questions separately. They filtered out all the skeptics, and continued the panel only with the subset who accepted their extreme position.
I genuinely felt like a heretic just for questioning whether superintelligent AI inevitably leads to the extinction of humanity, or entertaining the possibility that the future might actually turn out well.
That does not mean AI safety is fake. Obviously there are serious questions about rapidly advancing AI systems and what happens as they become much more capable. Those are exactly the conversations I went there hoping to have.
But what I encountered was an apocalyptic death cult. These people are not rational. They are deeply afraid, they reinforce that fear in one another, and they have become so emotionally committed to apocalypse that they refuse to be reasoned out of it.
Chris Mangum retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Chris Mangum retweeted
All the richest industry leaders agree:
There should be a conspiracy in restraint of trade, and the government should regulate their industry in the way they desire and are best prepared to comply with.
Chris Mangum retweeted
The biggest threat to Anthropic and its alleged $2+ trillion valuation is open-source AI.
We’re now entering the max fear mongering stage. Open-source AI could theoretically destroy enormous amounts of value in the snap of a finger.
The incentive is obvious. Protect the wealth, power, and valuations that have been created around centralized AI. This is a chess game to preserve that power and push valuations as high as possible, while preventing open source from throwing a wrench into the entire plan.
There is currently a massive PR effort to get the public behind government “protection” of these companies, under the guise of protecting the public.
Ultimately, centralized control over the most impactful technology in existence risks becoming a small group of people trying to play God.
Chris Mangum retweeted
Don't be fooled. We don't fear AI. Anthropic wants advanced ai classified as dangerous so only a few select government approved labs can work on it. They want to eliminate competition. They want to be the Northrop Grumman of AI.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
I think this is probably true and my observation as well
Theory: The difference in AI doomerism is largely generational, with the youngs thinking it's the end of the world and the olds not. Note how the 20-something lab staff thinks it's the techno-Rapture, and everyone from Jensen to pmarca (the olds) are telling them to chill out.
This is a symptom of a larger trend I've noticed as someone in the Xennial generation increasingly surrounded by Zoomers. There's what I call The Great Generational Inversion. The standard trope is that the conservative olds are scandalized by wayward youth, but it's precisely the opposite now.
The youngs don't offend me; I'd scandalize them if I actually said what I think. I'm not trying to censor them, they're (literally in my case, cf. Chaos Monkeys) trying to censor me. They don't seem wild and devil-may-care to me, they seem cautious and anxious and conformist. I'm still riding to hell on a fast horse lifestyle-wise, and they're all popping pills and peptides and as obsessed with their health as hypochondriac grandmas. The things the youth could always be depended on to do to excess--drinking, driving, fornication--they increasingly don't do at all.
It's very strange. Again, you could discard all this as the prattling of an out-of-touch boomer...but I'm not whining about our wanton, irresponsible youth, I'm saying they remind me of my grandparents and wish they lived a little more. It isn't how it's supposed to be, which makes it all the stranger.
Wonder if the generations will just all be flipped from the historical pattern going forward, or if this is a post-COVID one-off.
