Pinned Tweet
Honestly, why are we still obsessed with rolling dice? A single picture like this generates far more entropy.
Are we just thinking inside the box because Coldcard framed it that way?
Trustless protocol exist as we do not trust any party. If we do not trust RSI, then it is inevitably to use trustless protocol for every weights update.
Dario dream has always to achieve ASI first before China, so he can control them. If Dario RSI model is so good, he can easily hack and stop the chinese lab, without government intervention, but they cant, bacause their model doesnt ALIGN to it. Misalignment issue
KimLeong retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Well said. In fact, to chinese lab, instead of pacing AI, it signals them to accelerate further to mature its RSI.
Finally our AI leaders (Dario and @Sama) are doing the sane thing. "Pacing the frontier" is exactly the right thing to do: keep moving forward, coordinate the pace, and collaborate on alignment research. It is the right balance.
To the people whose objection goes to "but this hands the lead to China!" - no, YOUR ENTIRE WORLDVIEW IS GROSSLY WRONG - and moreover your ability to correctly perceive the world and predict events will always go awry until you correct it.
Here is the core of your gross error:
- You do not think people in China are actually people.
- You do not understand that Chinese civilization is a peer, if not superior, to your own.
If you actually thought of Chinese people as real people, with equally valid and real desires, dreams, and actual human existence equivalent to your own, you would realize that:
- China is not the Soviet Union, and you'd stop mapping notions of Stalinist authoritarianism onto it
- China's current government is valid and broadly supported by its people because it has improved the quality of their lives and continues to be focused on doing so
- Western values are not universal, as values are culturally influenced and ultimately validated by their consequences; the West has not been doing so well lately on that front, and so
- Your deontological bleating of "but we are free and they are not" is less than worthless: Chinese people most value being free from the actual entity most responsible for their oppression, namely WESTERN COUNTRIES
And so, if you can start to break out of the many layers of your brainwashing, you may further realize that:
If you are a country engaged in a geopolitical race to develop a key technology that:
1) may strongly determine your country's own future sovereignty,
2) one which almost every country agrees that if they don't have it, they will be subject to domination by other countries who do, and
3) your rival is clearly ahead, and
4) people are starting to realize that unrestrained development may lead to significant global danger,
then it should be perfectly logical that your answer to any call to "mutually agree to stop" must start with the player who is ahead agreeing first to slow down.
If you can realize that both countries are actual real civilizations with real human beings who have actual real lives who have exercised self-determination when it comes to how they are governed, and not One-Dimensional Cardboard Others, this is perfectly logical.
Imagine a scenario where the US and Soviet Union were engaged in a race to build ever more powerful and numerous nuclear weapons, and the Soviet Union was clearly ahead.
Then the Soviet Union points out that this risks worldwide nuclear annihilation, and proposes that both countries agree to stop building nuclear weapons and engage in mutual disarmament.
The obvious and logical reaction of the US would be to say, "Yeah, we agree there is a danger, but we'll agree to begin disarming once you've disarmed your own weapons until we're at parity."
Or, you'd try to accelerate your own development until the US reached parity, at which point you'd return to the table and have discussions about mutual disarmament.
Right? You definitely wouldn't agree to stop and disarm when the other guys were AHEAD. Your main danger is that the other guy is ahead, not something bad that lies ahead of them.
The argument in the US is "if we slow down, China will get ahead!"
Guess what, the real people in China see it as, "the US is already ahead! They're asking us to stop?"
The stronger party needs to make the gesture of good faith, if they are genuine.
There is real-life precedent to this!
Back in the day, when the US proposed to China that we engage in mutual nuclear disarmament, China's answer was basically "We will be happy to engage in mutual disarmament once the US first disarms to the same level of warheads that we have!"
For reference: China has ~600 nuclear warheads; the US has ~3,700.
If you are a country that was previously invaded and colonized by 8 countries at once, is surrounded by military bases held by your rival, whose neighbor committed horrific war crimes against your people and remains unrepentant and whose pacifism appears to only be maintained by that same rival occupying it... if you were a real human with a real life and real hopes and dreams, you would reasonably expect a superior rival proposing mutual disarmament to be the one to volunteer to stop first, if not disarm down to your level.
China currently has inferior models, lacks leading-edge chip technology, resorts to distilling US models to train its models. We have yet to see even a single Chinese model that does anything uniquely Chinese besides refusing to answer questions about Tiananmen. [1]
Recognizing that the people "on the other side" are also people means recognizing that they might also care about the same things you do and that they care about their lives and families and civilization and Not Getting Into A War Where Loved Ones Die and aren't going to do the "well a cartoon villain would do the evil thing so that must be what they would do" thing.
That is why your entire worldview is wrong. You think you are a real person and you don't think the other people are real people.
You think your country is filled with real people, and the other country that's much older and has four times as many people isn't filled with real people. So obviously you are not going to be able to form an accurate worldview.
Otherwise quite intelligent and thoughtful people in the US seem to have this blindness (it seems to be bipartisan), and don't really regard China as being a real civilization populated by real people who chose their government and want it to do what it's doing because it's actually good for them.
Here's a common "anti-safety-ist" argument you might've heard:
"The doomers are fearmongering and trying to make everyone scared of AI, just so they can justify a solution where they control everything and keep us from having abundance!"
Sound familiar? Do you believe it?
Then maybe you also believe the true version:
"Our leaders are fearmongering and trying to make everyone scared of China, just so they can justify a solution where they control everything and keep us from having abundance!"
If your reaction is "but no, China actually is... " maybe you want to realize how much you've bought into what you've been told. Maybe you're not aware of how many high-quality Chinese products you could affordably have access to. Do you know what abundance requires? Overcapacity.
The people telling you that are the same leaders who have been shown to lie to you about <oh man, just insert your own personal list>.
Why do you somehow think the one thing they're telling you the truth about is that China is a threat?
It's only a lie they can maintain because you readily believe that Chinese people in China aren't real people, because if you thought they were real people, fellow human beings, you'd realize that those lies could not be true. Chinese people are the most unruly, ungovernable people in the world. Chinese history is basically a history of overthrowing their governments - if they have a government that is popular it is because it is doing quite well for them - though it may seem weird to outsiders because Chinese people are pretty weird.
I know this, of course, because I'm ethnically Chinese and I know actual Chinese people, and at the same time I was raised and educated in the US so I know actual Americans, and everyone is Real People, not cartoon Cold War villains (who, when the Cold War ended, we learned were also real people with hopes and fears and dreams).
There is, overall, strong willingness to work together there - cynically, because working together is how you get rich, and Chinese people just want everyone to get rich; war does not make everyone rich.
But the US should stop being naive and realize that when it comes to AI it has to take the first step in extending the bridge. This announcement and the similar one from OpenAI a couple weeks ago are the right start.
====
[1] Seriously, where's the 5000 years of written training material? Why isn't Qwen quoting Confucian analects or traditional idioms to me when I ask it for advice? Why is it always neoliberal pablum? The US is more dominant in AI than it thinks.
Why do we think AI alignment is solvable when humans can’t even align with each other? @DarioAmodei @elonmusk @sama
Not a maxi, but i think @JasonPLowery softwar is talking about this exact threat. (only if P!=NP)
Seems to be promising but yet underdeveloped concept.
I heard rumour that Anthropic found a solution to AGI alignment issue. This seems to be relatively easier problem than the navier stokes
I spent much of the weekend talking with the team who did this work. Seb--and everyone else--acted with integrity and generosity throughout.
Initially we believed the other team had also solved the problem. We wanted to collaborate and do a joint release.
When we learned that they had Euler but not Navier-Stokes, we offered to let them go first, to suggest that they should be the ones to get the prize, and optionally for Tristan to be the lead author on a rewrite of the OpenAI proof. We felt it was challenging to offer the same to Levent (an Anthropic employee), who was not willing to talk or coordinate with us anyway. We were open to other solutions.
We would have greatly preferred coordination. We did not rush to publish even though the other team wasn't communicating with us. The team threatened us with unfounded accusations of plagarism.
Now that we can see their work, the approaches appear to be different. It is also worth noting that our latest model can solve many, many other math problems.
It is true that we tried this because there were rumors on the internet last week that Anthropic's models had solved a millennium problem and we were curious if ours could do it too.
Superintelligence alignment is just like human education. It is not possible to align every human interest through teaching alone. I think we need proper game theory applied to resource policy.
Asking human to QC agent output will be same as getting human to QC bot in chess game. This will be solved very soon.
In few months/years, can agent hack into bank ledger including its backups, and change the figure inside the ledger?
It is very possible we have more than one coldcard attackers now (anyone can attack now), since the cost of attack is cheaper than mining. Remove everything from your Coldcard even you have secured it with passphrase, do not reuse it unless it has been audited.
Stay safe
There are many random events in the world. Rolling dice just one way. Anyone can vibe code with old airgapped phone to produce real random seed without rolling dice
xitter.cf/ferenckovacs/status/20…
I tested rolling 6 dice 34 times, which is 204 rolls total. Theoretically, that gives about 357 bits of entropy, which in practice comes out to 256.
I tried it out, as you can see on the side, using the new firmware on the Coldcard Q with the update. It generates the exact same seed as the app.
I've never used these seed phrases for anything real, they're just for testing, so feel free to take them—there are plenty of them anyway.
So we can say that as a signing device, the Coldcard Q basically works like a standard. The 204 dice rolls provide plenty of entropy, and basically you don't need any manufacturer's device or any digital black box. Just a few dice and 20 minutes.
