@max_zuo

Ph.D Student @BrownCSDept | Generalization via Limited Supversion & RL | Advisors: @mlittmancs & @stevebach. Previously @gtcomputing & @google.

 Earth
maxzuo.ai
Joined November 2016
  • Tweets32
  • Following329
  • Followers297
7Media
Ever wonder if LLMs use tools🛠️ the way we ask them? We explore LLMs using classical planners: are they writing *correct* PDDL (planning) problems? Say hi👋 to Planetarium🪐, a benchmark of 132k natural language & PDDL problems. 📜 Preprint: arxiv.org/abs/2407.03321 🧵1/n
9
40
11
191
189,379
LLMs don’t learn or think like us. So why do we still evaluate their generalization in the same way? A better look at generalization in LLMs:
Yeganeh Kordi@yeganekordi29 Nov 2025
How well do language models generalize to problems that are harder, or even easier, than the ones they’ve trained on? We show that LLMs don’t generalize across difficulty levels quite as much as you might think. 🧵
Arxiv Submission: https://arxiv.org/abs/2511.21692

ALT Arxiv Submission: https://arxiv.org/abs/2511.21692

1
3
352
We're happy to announce that effective as of July 1, 2025, faculty members @stevebach and @drsrinathsridha have received named chairs. Steve is now the Eliot Horowitz Assistant Professor in CS and Srinath is the John E. Savage Assistant Professor in CS: cs.brown.edu/news/2025/06/02…
5
78
5,128
I started a blog! First post is everything I know about setting up (fast, reproducible, error-proof) Python project environments using the latest tools. These methods have saved me a lot of grief. Also a short guide to CUDA in appendix :) blog.apoorvkh.com/posts/proj…
6
8
71
5,475
If we guide the activation in the ‘right’ part of the subspace, we can improve performance pretty dramatically, although we don’t completely fix the problem.
1
3
4
445
🤔How do multilingual LLMs encode structural similarities across languages? 🌟We find that LLMs use identical circuits when languages share the same morphosyntactic processes. However, they involve specialized components to handle tasks if contain specific linguistic features⤵️
2
35
2
154
31,586
Is o1 really better at reasoning? Or is it just reinforcing what it already knows? We put o1-preview to the test on (some of) our planning problem generation dataset, planetarium🪐. Here’s what we found: 💻repo: github.com/BatsResearch/plan… 🧵Thread 👇
14
51
6
378
71,341
more replies
These characteristics (ignoring domain, better performance on common domains) seem more likely to be the fruit of an incremental improvement to training than a jump in reasoning, at least on our problems. 🧵4/5
1
1
42
3,858
o1 exemplifies more than ever a need to evaluate emergent reasoning capabilities separately from pretraining regurgitation, which is why we’re continuing to improve Planetarium🪐! 💻 Keep up to date through our repo: github.com/BatsResearch/plan… 🧵5/5

GitHub - BatsResearch/planetarium: Dataset and benchmark for assessing LLMs in translating natural...

Dataset and benchmark for assessing LLMs in translating natural language descriptions of planning problems into PDDL - BatsResearch/planetarium

github.com
2
2
51
3,631
Not only can't LLMs plan, they can't even generate specifications of a problem (in PDDL) that a standard planner could solve.
Max Zuo@max_zuo10 Jul 2024
Ever wonder if LLMs use tools🛠️ the way we ask them? We explore LLMs using classical planners: are they writing *correct* PDDL (planning) problems? Say hi👋 to Planetarium🪐, a benchmark of 132k natural language & PDDL problems. 📜 Preprint: arxiv.org/abs/2407.03321 🧵1/n
54
104
12
659
182,240
So, can LLMs actually “plan”? GPT-4o defers to using a "default" domain and ignores the one provided. Explicit problems are simple enough to be solved with simple copy and paste with regex. And no model does well on abstract problem definitions. (11/n)
1
1
9
1,623
This work was done in collaboration with the amazing @fpiedrahitav, @jacobli99, @mlittmancs, and @stevebach. Huge thanks to my fantastic co-authors! 🫶🥳 (12/12) 💻 Code: github.com/BatsResearch/plan… 🗃️ Dataset: huggingface.co/datasets/Bats…

GitHub - BatsResearch/planetarium: Dataset and benchmark for assessing LLMs in translating natural...

Dataset and benchmark for assessing LLMs in translating natural language descriptions of planning problems into PDDL - BatsResearch/planetarium

github.com
1
13
1,323
Load more