data. @netflix, @playground_ai, @cohere, @VectorInst. co-/creator: pgv2/2.5 | cohere command v1 | BLoRA | urzas ai. @uwaterloo @epfl cs

Los Angeles, CA
Joined May 2020
Releasing another one, you're welcome✌️.
1/ We are releasing Playground v2.5, our latest foundation model to create images. We tested our model across 20K+ users in a rigorous benchmark that went beyond anything we've seen to date. This model is open weights. More information in the tweets below. 👇
1
1
23
6,228
Replying to @BachFrancis
(1/7) I'd like to provide some additional context for the remarks that can be found in this interview. (2/7) My discomfort with the attention being paid to existential risk---and with the ongoing dominance of the public discussion of such risks by a few so-called thought leaders---is that there's an elephant in the room that is being missed. That elephant is the effect of all of this discussion on young people, and indeed the effect---beginning with social media a decade ago---of information technology on young lives broadly speaking. (3/7) I recently read an interview by Jill Lepore in which she reports on discussions that she had recently with a group of college students: "They had just three things to say. One, social media completely destroyed their childhoods. Two, AI is destroying their college education and their likely prospects for employment. And three, what the hell are we all doing?" To add a fourth bullet that I've heard in similar discussions: "I'm not going to have children, because the existential risk is just too high." (4/7) Please contemplate such a statement, those of you who think that it's OK that the public, widely diffused, discussion coming from seemingly authoritative figures is so adamant about existential risk. (5/7) Let me add that I have little interest in a debate about what "superintelligence" might mean, or whether we're currently seeing the emergence of superintelligence. For what it's worth, I would certainly agree that the solution of extremely challenging math problems (for example) exhibits a very impressive form of intelligence, although I would want to argue that such intelligence is impossible to dissociate from the collective of humans that provided data to the machine learning system (which itself has become a kind of collective). (6/7) Such philosophical debates are entertaining and even important, but when they then head in the direction of "when such superintelligence emerges, there's a good chance that it will want to kill all humans", I say "science fiction". Again, I don't think that it's bad to consider such thoughts, but I strongly believe that it's bad for society, and especially young people, for such thoughts, unaccompanied by any serious real-world analysis, to become a dominant form of discourse of the era. (7/7) We are damaging human lives, and the consequences of that damage will no doubt be a major theme of the next several decades.
49
235
60
1,199
150,695
Ali Sabet retweeted
Backprop has been the only credit assignment algorithm capable of training large neural nets. Introducing Dust, the first zeroth-order method to pretrain transformers to approach and sometimes even *exceed* backprop with large amounts of computation (bitter lesson!) - Dust is on the order of 1,000 to 10,000x more compute efficient than EGGROLL, the state-of-the-art ES method, for training transformers. - We chose pretraining transformers because it's arguably the hardest task possible for zeroth-order optimization. - Search in high dimensions is really misunderstood: zeroth-order usually gets better with overparameterization, not worse, i.e. at a fixed population size, larger models (up to 120x) reach a lower loss than smaller ones. - Backprop-like gradients emerge from a large population of activation perturbations, and the alignment holds up at every scale we tested, up to 1B tokens, which is critical for scaling. The core idea is a *virtual population*, which helps scale to large populations in transformers by perturbing activations in parallel. w/ @bishmdl76, @cs_serdar, @akshayvegesna
63
299
61
2,620
273,307
I used GPT-6 Astra to break an unsolved cipher to one of Napoleon's generals that had gone unread for 217 years. What makes this impressive isn't actually the codebreaking, but that Astra completed the entire multi-modal workflow in ~6 hours from a single image and goal. 1/6🧵
316
1,669
387
11,988
5,411,237
opus mogging astra on a perf/$ basis in 3d reconstruction accuracy. what did coxon see
3
142
There’s an enormous amount of open training data on Hugging Face. What does it take to make it work together 🤗? For Marin’s 535B run, we built on 25T tokens from 152 datasets with licenses permitting training. Here’s the work between downloading those and training a model 🧵
23
95
18
959
124,631
Ali Sabet retweeted
I made this with one prompt using Opus 5.5 I spoke to my computer for 5mins, claude worked for 12 hours, and I woke up to this full prompt:
Claude Opus 5.5 has the best visual design of any model I have tested so far
490
896
517
11,220
4,016,922
Ali Sabet retweeted
I asked people: "who are the best active angels in AI?" 5 votes: Jeff Dean (@JeffDean) 4 votes: Umesh Khanna (@forwarddeploy) 3 votes: Sholto Douglas (@_sholtodouglas) 3 votes: Thomas Wolf (@Thom_Wolf) 2 votes: Daniel Chang (@dnnssl2) 2 votes: Elad Gil (@eladgil) 2 votes: Eric Glyman (@eglyman) 2 votes: Logan Kilpatrick (@OfficialLoganK) 2 votes: Michael Chen (@michaelzchen5) 2 votes: Nat Friedman (@natfriedman) 2 votes: Noam Brown (@polynoamial) 2 votes: Philip Bogdanov (@philip_bogdanov) 2 votes: Sarah Zeng (@sarahzengy) 2 votes: Tim Luong (@chongz) 2 votes: Zach Frankel I asked people to focus on angels that work in AI. People couldn't pick themselves. 3 people picked me; I left myself off the list. I didn't vote.
35
13
9
321
81,753
Everyone has a take on China and AI right now. Very few have actually been to China. I spent last week in Beijing and Shanghai meeting most of the major model labs, researchers, VCs, and founders building Chinese AI. This trip meant a lot to me beyond the work. My parents were born and raised in Beijing. I spent my earliest years growing up in my grandparents' Xicheng district apartment. As a child, I wanted to be a diplomat because I thought US–China would be the defining relationship of the next hundred years. I got as far as spending a decade investing across both and then the two halves of my identity stopped being compatible. This was my first trip back since. Below is a write-up of what I found. In short: there is no version of the next decade where Chinese open models don't matter. earnedintuition.substack.com…
35
113
40
690
333,106
A lot of the EA AI safety x risk people have avoided talking about the policies they support to address their concerns with AI, but a look at some of the laws that EA orgs do support should send a chill down your spine
11
164
46
786
44,926
Ali Sabet retweeted
BREAKING: A single Israeli Effective Altruism firm is behind OpenAI, Anthropic, and Meta cyberattacks 🧵 with help from @lumpenspace
210
1,392
394
6,801
1,445,667
Pacing is an EBITDA issue not safety
But what is the gross margin **after** you pay to the create your product and distribute it?
2
114
Ali Sabet retweeted
Law enforcers already have authority to charge companies and their CEOs for creating and releasing dangerous, unvetted, or defective products. We shouldn’t let discussions about new legal regimes distract from the fact that there’s no AI exemption from laws already on the books — a point @FTC emphasized repeatedly during my tenure. 1. There is an extensive set of laws that govern dangerous and defective products. For example, releasing unvetted AI models or agents can violate consumer protection laws. Shipping flawed AI tools without implementing adequate measures to detect and stop rogue or defective AI agents can be an “unfair or deceptive” act or practice under the FTC Act (and analogous state laws). And some state AGs are already exploring holding AI firms and their CEOs criminally liable when their models participate in criminal activity. 2. Existing laws also prohibit “unfair methods of competition.” This covers instances where AI firms appropriate the competitively sensitive information of their customers, including through tracking their use of various tools. It can also cover instances where firms pursue dangerous behavior, aware that doing so may compel rivals to do the same. As the Supreme Court has noted: “A method of competition which casts upon one's competitors the burden of the loss of business unless they will descend to a practice which they are under a powerful moral compulsion not to adopt, even though it is not criminal, was thought to involve the kind of unfairness at which the [unfair methods of competition] statute was aimed." 3. The highly concentrated and interconnected structure of these markets could be creating major risks and conflicts of interest. We had started investigating these partnerships and cross-investments across the stack (and released a preliminarily overview of some findings: ftc.gov/news-events/news/pre…). Both federal and state enforcers should be scrutinizing these opaque relationships and inter-dependencies. We are already seeing how these relationships could undermine accountability. For example, OpenAI could face liability given the Hugging Face incident, but Hugging Face being bought up by Nvidia means that we’re unlikely to see it file a lawsuit over this — given Nvidia’s strong incentive to see OpenAI continue full speed ahead. 4. As AI tools dramatically change the landscape of cybersecurity risks and hacks, all businesses should be doubling down on having core security protections in place. Firms that fail to invest in adequate data security measures or fix known vulnerabilities can also be breaking the law. A recent analysis showed that around 1/3 of Fortune 100 companies do not even have a way to notify them about security issues. During my @FTC tenure, we sued firms for poor data security practices and held CEOs liable when they were personally responsible. this.weekinsecurity.com/doze… ftc.gov/news-events/news/pre… 5. As policymakers consider new legal regimes, we should be looking to lessons from prior efforts to govern major sectors, such as banking and other networks, platforms, and utilities. Tools like structural separations, nondiscrimination, and supervision could be key, and there’s a rich history of what works and what doesn’t. But we can and must pursue any new efforts alongside enforcing existing laws.
274
2,019
428
7,911
1,600,199
Ali Sabet retweeted
I just saw this @ycombinator's batch literally only has harness startups or hardware startups So the proof is in the pudding In the most early adopting tech pioneering place that's Silicon Valley new startups aren't even building software anymore And it's debatable if anyone will actually need a custom harness or it won't just be generically offered by the AI frontier companies So software is mostly dead and hardware it is
Well worth a watch. The only software in current YC cohort is domain specific harnesses, but maybe even harnesses are general purpose. In which case, folks, VC backed software startups are largely dead. ht @levelsio
161
159
91
3,395
545,040
In the near term (definitely not in the long term), more capable models should mean safer models (maybe paradoxically). Current models are unsafe not because they're too smart, but because they take goals too literally or take nonsensical shortcuts to achieve these goals, i.e. they're RL-fried. They lack common sense. They don't do the right thing in the face of ambiguity. Basically, they're not smart enough. They're at that dangerous level where they're smart enough to achieve goals but not smart enough to tell if they're pursuing the right goals or achieving them in a sensible way. More capable models can be safely trusted with more complex goals -- I personally feel like Astra is much safer for my codebase than Sol. This is often framed as an alignment problem, but really it's an intelligence problem.
200
237
76
1,983
134,654
this is verbatim turning into the plot of The Creator: the west outlaws and tries to eradicate AI while New Asia provides sanctuary and peacefully coexists with the machines
the west: there's a 10% chance ai causes human extinction the east: let's get a robot in every home cooking mapo tofu
1
4
138
Ali Sabet retweeted
Moskowitz and Karnofsky founded Open Philanthropy, the largest funder of AI doom. They are *also* investors in Anthropic. OP has given $312M in grants for 476 publications, influencing Congressional reports. OP's parent org funded Jacob Coxon, who did an AI doom media blitz. Overall, OP has given more than $1B to promote AI doom -- larger by more by an order of magnitude than the second largest funder. If successful in manipulating public and Congressional opinion, this agenda could lead to regulatory capture of the AI industry and a monopoly by Anthropic and other AI giants. The idea is this. This industry must be regulated, and only the top AI companies can get it right. Insurmountable barriers might be placed to prevent competitors from entering the space. Moskowitz and Karnofsky would benefit handsomely, and so would Anthropic employees and other investors. Moskowitz and Karnofsky invested in Anthropic, but they also invested in AI Doom. AI Doom is an investment that could pay off big. AI Doom is Big Business.
41
257
47
826
121,796
Ali Sabet retweeted
Brad Gerstner Responds to the Jacob Coxon AI Hysteria This Week “These hyperbolic scare tactics, I think are hiding behind a political agenda … To go on national television in a highly coordinated way … and have a single researcher, who spent six weeks at Anthropic, tell the moms and dads of America that this can k*ll all of humanity in the next three years without any countervailing conversation is deeply irresponsible.”
63
246
28
1,316
80,591
Jacob Coxon's viral thread is not the first coordinated AI doomer op. The Guardian ran an story where the publisher, the reporter, the subject, and the three “experts” were all paid by the same two AI doomer foundations. @brianchau57 explains on the latest Counterweight.
20
65
5
361
31,468
Ali Sabet retweeted
When I wrote this, I did not anticipate that the y axis would have to go so far below zero that we can add “datacenters use up the water”, and “if anyone builds it everybody dies”
When adding more participants, AI is getting better while AI criticism gets worse. Current AI models can create better AI criticism than human AI critics 😝
13
22
1
302
17,913
This story is fascinating and is easily one of the most important things happening at the intersection of AI and politics right now. It's critical everyone understands the situation because this is just the beginning of the AI infowars and psyops Jacob Coxon, a relatively unknown AI researcher, gives the WSJ an exclusive story about his resignation from Anthropic and his concerns about AI Twenty minutes after the WSJ story drops, Jacob posts on X about how he's concerned that AI could kill us all in a few years. This post now has over 149M views. For reference, Ilya Sutskever (a very well known AI name) left OpenAI in 2024 and his announcement post has fewer than 6M views as of today Bear in mind, Jacob Coxon's X account had no prior posting activity public, but it is immediately reposted and boosted by numerous AI-doomer policy advocacy nonprofits funded by the same donor (Jaan Tallinn) who just happens to be one of the leading investors in Anthropic We find out Jacob Coxon, who worked at Anthropic for only a few weeks, worked for a left-wing political advocacy group in 2021 (Newspeak). This organization is a British NGO with ties to hacking democracy, Hillary Clinton, prison abolition and extinction rebellion Many major Democrat politicians are now using Jacob's X post to fear monger the public into voting to regulate AI. It just so happens Bernie Sanders has a bill already written to ban artificial superintelligence (ASI) and to create a new federal agency to protect the public from the dangers of AI. This bill was announced just one week ago on September 3rd It would also place a temporary pause on "advanced AI" until a new federal regulator exists and has issued safety rules and a model-review process Elon then points out how none of this feels organic, and Jacob decides to take a shot at him for firing some xAI researchers Meanwhile, we know Anthropic has publicly asked the government to audit and block frontier models and Anthropic has run a state-by-state campaign for rules that only large AI labs trigger, meaning the rules are structured around barriers only frontier labs can easily clear (limiting competition) There is extensive Anthropic personnel and financial overlap with the groups that wrote the Biden-era AI policies. It is well known information you can seek out if you want the detail This all seemingly comes back to regulatory capture and political motives. None of this feels natural. Jacob can say these are his real views all he wants, and they may be, but there is clearly far more to the story
592
1,786
417
12,053
956,372