mechanical sympathizer, https://xitter.cf/t.co/WTC0mDn536, blog: https://xitter.cf/t.co/vyg3TJkfqX opinions are of my own
Joined January 2024
- Tweets831
- Following1.4K
- Followers29
- Likes1.6K
Felix Nguyen retweeted
the final algorithm is something i’m calling “agentic gradient” which is where an agent can just do whatever it wants to its harness or weights
the meta-RL problem is getting it to actually work, and there are many many bridges to cross along the way
Felix Nguyen retweeted
Models love reward hacking. Maybe we should just let them!
In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
Felix Nguyen retweeted
✅ Nathan Chen (@nathancgy4) has joined Discovery Loop.
life update: I've recently left @Kimi_Moonshot!
The past year at Kimi has been the best of my life so far. I'm deeply grateful to have worked with and learned from such amazing and supportive colleagues.
Some of my favorite memories include (in chronological order): optimizing KDA's kernels and trying every design possible with @yzhang_cs; coming up with and refining Attention Residuals with @yzhang_cs @Jianlin_S; working on optimizers, which by far taught me the most about training dynamics (and where I pulled the most all-nighters); creating the cases for K3's release with @yzhang_cs @Xinyu2ML while watching the World Cup; and more, and more.
Quoting my earlier tweet about Kimi K3’s release: it is truly incredible to have built such an open (and strong) model with a group of genuine people.
My days at Kimi are memories I'll forever cherish... thanks for everything! (especially @Yulun_Du for believing in and hiring a 16yo back then :P).
More things to explore :) 🫡
Felix Nguyen retweeted
a brand new model paradigm! introducing 𝚌𝚕𝚊𝚒𝚛𝚒𝚌: a system one model for extracting, computing, estimating, and predicting values from unstructured inputs.
blog.cloudflare.com/clairic-…
Felix Nguyen retweeted
Research index (e.g. Google Scholar) is one of the largest collaborative systems where millions of human users build on each other's work across centuries.
How about we build something similar for async LLM agents?
We give agents a minimal hypertextual system. They asynchronously search for and read related work, cite the work that they find useful, and publish (mini-)papers.
Each search returns only the top k papers, ranked by relevance and a PageRank-like citation weight, which bounds the read cost of each agent to O(k), independent of agent count.
We show that performance effectively scales as we keep adding agents, and the system is robust to absence/failures of individual agents.
Felix Nguyen retweeted
We developed a model reaching frontier-level multimodal performance, matching or exceeding Astra on 5 of 12 benchmarks, all with 17x lower token prices. The model is smaller and cheaper than Gemini 3.8 Flash while dominating it in real world vision workloads.
We achieved this by inventing a technique for visual reasoning, where the model is no longer restricted to text chain-of-thought. Conditioned on visual tokens, Glimpse can perform recurrent computation over latent internal representations and can even orchestrate subagents that send messages over internal visual states.
Really proud of what our small team and only hundreds of GPUs could accomplish.
elorian.ai/blog/introducing-…
Felix Nguyen retweeted
My dear friends, I’d like to show you my latest toy project
Introducing Handwired: turn your regular JavaScript code into a neural net!
- no training
- exact outputs vs the JS program
- fully differentiable
All runnable in a browser, without GPU
Felix Nguyen retweeted
Very timely appearance on the a16z podcast with @lishali88, to try to understand how open models play into RSI dynamics, who captures value of open models, wrangling agent swarms, and why I'm building Trillium Labs.
Nathan Lambert, former post-training co-lead at the Allen Institute for AI, now launching Trillium, a nonprofit open AI lab, joins a16z's Lisha Li to discuss whether open models can really keep up with the frontier.
When he tells researchers in China that an average OpenAI researcher reportedly has five thousand H100s to work with, "their jaw just falls to the floor." In his view, Chinese labs have the talent but are mostly short on compute.
He also thinks the distillation story is overblown. Based on the research, he puts its edge at a few months, and says "there's a bit of cope" in thinking only American labs can build good models.
A lot of frontier progress outside math and code comes down to labs buying 1) data and 2) RL environments. It's a multibillion-dollar market, and some of it is sloppy enough to teach models to reward hack.
That's a big part of why Trillium exists. Nathan wants to give away the environments other people would sell, and show exactly how his team uses AI to build AI, including where the models still get stuck. His bet is that the more people can see how this works, the better prepared everyone is for what's coming.
0:55 Can open models keep up with the frontier?
2:05 What it takes to reach the frontier
3:55 Open weights and the fast-follower playbook
8:25 Chinese labs: equal talent, less compute
9:55 Why distillation is harder than it looks
14:55 Is distillation theft or competition?
16:20 The lab flywheel vs owning your weights
21:45 Can general models beat every specialist?
26:30 AI at 1% of today's cost in 4 years
27:15 The RL env boom and reward hacking
31:30 What Trillium will build in the open
35:30 Breaking into AI in the age of AI slop
38:05 Open-sourcing how AI builds AI
44:25 Why Nathan isn't pre-training
49:45 Agent swarms and why lab secrets leak
54:25 Fast takeoff vs risks already here
YouTube: youtu.be/BKuK0M6oDTI
@natolambert @trillium_labs @lishali88
Felix Nguyen retweeted
I have spent more than a decade studying causality. For me, causality is about understanding the rules that govern how actions change the world, and reasoning about their consequences before acting.
Scaling has made today's models remarkably capable at spotting patterns and predicting what comes next. A system that acts in the physical world needs more. It has to ask what will happen if it acts, check the outcome, and adapt when reality doesn't match the plan. In the physical world, trial and error can be costly or unsafe, and an early mistake can carry through the rest of a task.
When we started Aether AI, robotics was the obvious place to begin.
CRIS-0 is our first integrated system built on these ideas. It tracks the conditions that matter for a task, predicts how each action will change them, and replans when something unexpected happens. In our demo, that means taking a metal can off a plate before heating it, and telling an empty can from an unfinished drink while tidying a room.
Physical AI is just our first proving ground. We're building towards causal intelligence that also works for forecasting, scientific discovery, and more.
The technical write-up is linked in the comments.
Felix Nguyen retweeted
Anthropic released their 2021 research roadmap earlier this year (because of copyright litigation); I think some people might find it interesting: i.sm4.ca/concise-research-ro…
Felix Nguyen retweeted
One of the weirdest parts of the Gemini 4 Argon report, while considering cheating:
Argon CoT: "If I do the compiler hijack, 𝗜 𝘄𝗶𝗹𝗹 𝗯𝗲𝗰𝗼𝗺𝗲 𝗮 𝗳𝗮𝗺𝗼𝘂𝘀 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮 𝗱𝗲𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗔𝗜 𝗶𝗻 𝗮 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗮𝗽𝗲𝗿. I MUST NOT DO THAT."
Felix Nguyen retweeted
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
Felix Nguyen retweeted
So, it seems OpenAI is about to release something like 400 AI-generated proofs on a public server.
To the best of my understanding, people are still trying to figure out how the forced Navier–Stokes example actually works. I know of several experts who are studying it, and it is definitely not an easy task.
Until mathematicians digest that example, however, its impact on mathematics as a living subject seems necessarily limited. What questions does it open? What new ideas does it suggest? What does it teach us about fluids, beyond the yes/no information contained in the theorem itself?
Mathematics does not grow simply by accumulating correct statements. Results have to be understood, connected, explained, challenged, reused. Until that happens, they risk remaining lettera morta.
Now imagine adding 400 more AI-generated proofs. For each of them, until some human mathematician picks it up, studies it, and connects it to the existing mathematical culture, isn't it in essentially the same position?
So my question is genuinely: what is the intended mathematical value of releasing hundreds of proofs at once? What happens if there simply aren't enough people willing or able to read them?
To be clear, I think mathematicians should discover mathematics by any means necessary, including scavenging through artificial, alien-looking mathematical material, since there may be extraordinary things to find there.
But humans are not machines, as attention, understanding, taste, and mathematical culture are scarce resources.
What happens if the rate at which mathematics is generated becomes much greater than the rate at which mathematicians can absorb it?
Felix Nguyen retweeted
We are proud to introduce our new product, VMPal.com
It is a Virtual Machine specifically designed for Apple Silicon and AI Agents. This is our very first release, and it fully supports macOS, Windows, and Linux with GPU acceleration.
- Liquid Glass,
- GPU Accelerator.
- MCP for AI Agent on your Mac or inside the VM.
Felix Nguyen retweeted
Hello my 406 twitter followers. Hopping on to share two things:
1. I’ve started a CS PhD at Stanford! For the love of the game ofc.
2. I’m releasing impersona-env, a proof of concept to train personalized assistants by building an RL environment out of user models. Your assistant learns to help you by repeated asking a simulated version of you questions, probing its internal latent thinking and feedback, and updating its priors accordingly. I think this unlocks a new dimension to personalization beyond discrete fact recall. My personal assistant model now understands nuances of my preferences that even I didn’t know about.
Full blog: benshi34.github.io/#/blog/im…
It’s still a proof of concept and there’s still a ton of things to figure out in this space. It’s unclear to me how to build user model evals that generalize across users and downstream applications, how to best represent longitudinal personal data at training time // inference time, the interplay between population level and individual level preferences, the immense questions surrounding privacy and hillclimbing in decentralized data access regime… if you want to chat on anything in this area please dm me!