
building neural interfaces @Meta @RealityLabs, prev: cognitive science @nyuniversity @Rutgers_Newark @UniOfAdelaide
New York City
Joined April 2009
- Tweets52
- Following1K
- Followers1.2K
- Likes4.1K
Pinned Tweet
1/ Today in Science, we train a neural net from scratch through the eyes and ears of one child. The model learns to map words to visual referents, showing how grounded language learning from just one child's perspective is possible with today's AI tools. science.org/doi/10.1126/scie…
Wai Keen Vong retweeted
New ARC-AGI paper
@arcprize w/ fantastic collaborators @xu3kev @HuLillian39250 @ZennaTavares @evanthebouncy @BasisOrg
For few-shot learning: better to construct a symbolic hypothesis/program, or have a neural net do it all, ala in-context learning?
cs.cornell.edu/~ellisk/docum…
Wai Keen Vong retweeted
Exciting episode! 🎙️ We chat with Dr. Wai Keen Vong (@wkvong), lead author of the Science paper where an artificial neural network mapped words to referents using a child's egocentric recordings. Plus, we dive into Wai's journey in science!
share.fireside.fm/episode/R8…
Wai Keen Vong retweeted
We have a new preprint led by @solimlegris and @wkvong (w/ @LakeBrenden ) looking at how people solve the Abstraction and Reasoning benchmark. arxiv.org/abs/2409.01374 ARC is one of the most challenging & long standing AGI benchmarks and the focus of a large $$ prize @arcprize 🧵
Wai Keen Vong retweeted
One inspiration for ARC-AGI solutions is the psychology how humans solve novel tasks.
A new study by @todd_gureckis @LakeBrenden @solimlegris @wkvong @ NYU explores human performance on ARC, finding that 98.7% of the public tasks are solvable by at least 1 MTurker.
Researchers at NYU did a study on whether MTurkers could solve ARC-AGI tasks. They found that 790/800 (98.7%) of the public tasks are solvable by at least one MTurker (each task was seen by about 10 people): arxiv.org/abs/2409.01374
As a reminder, the private test set was independently checked by 2 STEM professionals, who each solved 97-98% of the 100 tasks, with 100% of tasks solved across the two of them.
Wai Keen Vong retweeted
How do we measure similarities and differences between how kids and models learn language?
We introduce DevBench, a suite of multimodal developmental evaluations with corresponding human data from both children and adults. 🧵 1/
📝: arxiv.org/abs/2406.10215
Wai Keen Vong retweeted
Fun (and science) with the family highlighted in today's NYT. Comments from some of my favorite scientists @cocosci_lab, Linda Smith, and @mcxfrank whose study Luna is participating in. Thanks @oliverwhang21 for the great writing. nytimes.com/2024/04/30/scien…
Wai Keen Vong retweeted
Linda Smith, one of the scientists I most admire, writes in Nature "News and Views" about @wkvong 's work. Linda's conjecture: "problems of data-greedy AI could be mitigated by determining and then exploiting the natural statistics of infant experience" nature.com/articles/d41586-0…
1/ Today in Science, we train a neural net from scratch through the eyes and ears of one child. The model learns to map words to visual referents, showing how grounded language learning from just one child's perspective is possible with today's AI tools. science.org/doi/10.1126/scie…
Wai Keen Vong retweeted
In just a minute, @wkvong is being interviewed on NPR Science Friday, about training models through the eyes and ears of a child. listen in here: kunc.org/
Also, see the NPR link here: sciencefriday.com/segments/l…
Wai Keen Vong retweeted
My thoughts on the fascinating new study in Science
science.org/doi/10.1126/scie… by Vong et al in the Financial Times. 1/2
ft.com/content/f671968a-5750…
Wai Keen Vong retweeted
Excellent article on @wkvong 's new work in the Washington Post by @Carolynyjohnson, with discussion from Josh Tenenbaum and Michael Tomasello. Indeed, the next q is how verbs, pronouns, and abstract words can be learned. How far will data-driven go? washingtonpost.com/science/2…
Wai Keen Vong retweeted
Love the science.org home page right now
(this isn't the baby who participated in the current study, but in the next one :))
Wai Keen Vong retweeted
What can an AI model learn when given the data from just one child?
A surprising answer comes from a groundbreaking study published in Science by CDS Research Scientist Wai Keen Vong (@wkvong) and CDS Assistant Professor Brenden Lake (@LakeBrenden).
nyudatascience.medium.com/wh…
Wai Keen Vong retweeted
Published in Science today, @wkvong reports a dream experiment: he trained a multi-modal AI model from scratch on a subset of one child's experiences, as captured by headcam video. Shows how grounded word learning is possible in natural settings, as discussed in his thread:
1/ Today in Science, we train a neural net from scratch through the eyes and ears of one child. The model learns to map words to visual referents, showing how grounded language learning from just one child's perspective is possible with today's AI tools. science.org/doi/10.1126/scie…
10/ Finally, our findings address a classic long-standing debate in philosophy and cognitive science: What ingredients do children need to learn words? Given their everyday experience, do they (or any learner) need language-specific inductive biases or innate knowledge to get going? Or can joint representation and associative learning suffice? Our work shows that we can get more with just learning than commonly thought.
11/ A special thanks to my fantastic collaborators: Wentao Wang (@wentaow10), Emin Orhan and Brenden Lake (@lakebrenden), as well as all of the reviewers and commenters who provided feedback on earlier versions of this work. We're very excited to finally share this research!

















