ℤ
Joined July 2021
Live today on Symbiosis, trade blocks of @Polymarket without moving the books.
7
8
1
15
549
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
885
2,634
1,442
19,829
4,554,637
Great collaborating with our friends @transparentspc on visualizing the nth order interactions of related prediction markets.
1/ Almost anything that happens on a football pitch can now move a market. In the England–Argentina World Cup semi-final, Gordon’s goal sent England’s win odds from 28% to 66% in one second. Qualification odds followed 3 seconds later; title markets took 20–30s 🧵
7
5
18
1,564
selling data to labs is the new dropshipping
40
41
8
962
80,311
Seeing this evolve over time from some of the early simulation efforts is inspiring and excited. Amazing go getters and builders behind this.
1/ Our team at Anthias Labs is excited to announce the beta release of Arena: a robust LP simulation framework by @ts0yu0 via the Uniswap Community Research Program.
3
7
1,486
1/ Our team at Anthias Labs is excited to announce the beta release of Arena: a robust LP simulation framework by @ts0yu0 via the Uniswap Community Research Program.
1
8
2
32
5,054
Believe it or not, the name Strawberry does not come from the “How many r’s are in strawberry” meme. We just chose a random word. As far as we know it was a complete coincidence.
75
40
13
1,081
109,929
the efficient market hypothesis is interesting not only because it’s widely believed and obviously untrue, but also because it becomes more untrue the more people act as if it were true.
502
565
191
5,521
1,400,816
What we're building: 148th intent-based 5d solver rfq censorship resistant relay agnostic game theory optimal auction model with 7ms better latency than current sota What people want: 100x-DOGE-SHIB-PEPE-WIF-SLERF index
2
7
1
40
4,275
Parallel EVM claim, how to verify the claim of parallel execution. Wrote on this here: rakita.github.io/blog/blog/p…
6
27
1
115
19,119
I mean if the end is inevitable, why not just get there faster
3
5
64
3,274
Zach retweeted
human level ai is boring. please give vast datacenters drawing most of civilizations energy needs calculating inscrutable mysteries where not even the problem statement, much less the solutions, would make sense to mortal minds
54
36
10
512
47,058
Replying to @natfriedman
the path forward is: - discard all 'alignment' '''work''' - recast the field as controls and interpretability - get some people who know what a dynamical system is working on the problems good chance to unlock some pretty interesting stuff!
6
13
3
122
10,558
Zach retweeted
to enter the server you must produce a token of -21 logprob
31
5
2
312
41,979
sorry if I can't rebuild the universe from first principles using your primitive i don't want it
4
1
33
2,467
Here is 4-bit inference of LLaMA-7B using ggml: github.com/ggerganov/llama.c… Pure C/C++, runs on the CPU at 20 tokens/sec (M1 Pro) Generated text looks coherent, but quickly degrades - not sure if I have a bug or something 🤔 Anyway, LLaMA-65B on M1 coming soon!
24
118
29
704
116,039
why do most approaches to controlling model behavior rely on acting only in the model's input domain? prompt engineering, fine-tuning etc. all feel like trying to pick a lock with a wet noodle.
7
7
1
59
12,370
brains consistently having having hemispheres is pretty sus. it might be proof that having smaller latent space interfaces between different multi-modal encoders/decoders is a huge computational advantage maybe why text-to-image models get away with being so darn small
4
3
32
2,670
We trained a transformer to grok modular addition and reverse-engineered it. We found that it had learned a Fourier Transform and trig identity based algorithm, so cleanly that we can read it off the weights! I did not expect this algorithm! I found it by reverse-engineering.
3
23
7
117
14,397