Zach retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Zach retweeted
Great collaborating with our friends @transparentspc on visualizing the nth order interactions of related prediction markets.
Zach retweeted
Includes websocket support, clob interactions, and more! Now you too can trade on the highest volume : worst docs infrastructure in the world!
github.com/nhtyy/polymarket-…
Zach retweeted
Seeing this evolve over time from some of the early simulation efforts is inspiring and excited. Amazing go getters and builders behind this.
1/ Our team at Anthias Labs is excited to announce the beta release of Arena: a robust LP simulation framework by @ts0yu0 via the Uniswap Community Research Program.
Zach retweeted
1/ Our team at Anthias Labs is excited to announce the beta release of Arena: a robust LP simulation framework by @ts0yu0 via the Uniswap Community Research Program.
Zach retweeted
Believe it or not, the name Strawberry does not come from the “How many r’s are in strawberry” meme. We just chose a random word. As far as we know it was a complete coincidence.
Zach retweeted
the efficient market hypothesis is interesting not only because it’s widely believed and obviously untrue, but also because it becomes more untrue the more people act as if it were true.
Zach retweeted
What we're building: 148th intent-based 5d solver rfq censorship resistant relay agnostic game theory optimal auction model with 7ms better latency than current sota
What people want: 100x-DOGE-SHIB-PEPE-WIF-SLERF index
Zach retweeted
Parallel EVM claim, how to verify the claim of parallel execution.
Wrote on this here:
rakita.github.io/blog/blog/p…
Zach retweeted
Replying to @yacineMTB @suchenzang
I mean if the end is inevitable, why not just get there faster
Zach retweeted
human level ai is boring. please give vast datacenters drawing most of civilizations energy needs calculating inscrutable mysteries where not even the problem statement, much less the solutions, would make sense to mortal minds
Zach retweeted
Replying to @natfriedman
the path forward is:
- discard all 'alignment' '''work'''
- recast the field as controls and interpretability
- get some people who know what a dynamical system is working on the problems
good chance to unlock some pretty interesting stuff!
Zach retweeted
sorry if I can't rebuild the universe from first principles using your primitive i don't want it
Zach retweeted
Here is 4-bit inference of LLaMA-7B using ggml:
github.com/ggerganov/llama.c…
Pure C/C++, runs on the CPU at 20 tokens/sec (M1 Pro)
Generated text looks coherent, but quickly degrades - not sure if I have a bug or something 🤔
Anyway, LLaMA-65B on M1 coming soon!
Zach retweeted
why do most approaches to controlling model behavior rely on acting only in the model's input domain?
prompt engineering, fine-tuning etc. all feel like trying to pick a lock with a wet noodle.
Zach retweeted
brains consistently having having hemispheres is pretty sus.
it might be proof that having smaller latent space interfaces between different multi-modal encoders/decoders is a huge computational advantage
maybe why text-to-image models get away with being so darn small























