Jay Shim retweeted
Very excited to share that our CRL-VLA paper has won the outstanding paper award at the Reinforcement Learning Conference (also best paper at the ICRA RL4IL workshop)! The full list can be found at rl-conference.cc/RLC2026Awar….
This is my first time attending @RL_Conference, and RLC has already became my favorite conference (alongside CoRL). The community, the great review process, the hot takes... Everything is amazing!
Congratulations to all co-authors: @jayjshim @ChenTangMark @yoonchangsung @cranialxix @PeterStone_TX @RobobertoMM
VLA models are capable generalists. But can they continually self-improve?
Such Continual Reinforcement Learning (CRL) problems are traditionally considered very challenging.
Surprisingly, we found that with the right setup, the simplest CRL recipe can work really well!
arxiv.org/abs/2603.11653
Jay Shim retweeted
VLA models are capable generalists. But can they continually self-improve?
Such Continual Reinforcement Learning (CRL) problems are traditionally considered very challenging.
Surprisingly, we found that with the right setup, the simplest CRL recipe can work really well!
arxiv.org/abs/2603.11653
Realizing that building AI for healthcare means first working on safety reshaped how I think about my path.
@DarioAmodei's essay on powerful AI left me with both awe at what's coming and urgency about getting it right, so I wrote down my thoughts.
shimboi.hashnode.dev/the-rea…
On my TODO list is figuring out how to get multi-node pytorch training working with ray, FSDP, Huggingface, etc. Will keep you updated on my progress
Anyone have some comprehensive resources for learning to use Claude Code? I've been seeing it everywhere on my feed and excited to dive into it
TIL: Forcing the model to output two tokens at opposite extremes for the gripper dimension doesn't destabilize/make it more difficult for the model to learn even though it's bin size is >> 2. Feel free to let me know if you've seen instances that disagree
Spent a few days debugging a policy and found that the dataset I'm training on requires the gripper dim to be negated and spread out to {-1,1}. Hopefully this helps at least one other person since it was hard for me to find the 3 lines of transformations in a giant codebase
TIL: the LIBERO dataset suites have image observations that are upside down (flipped over y axis). I guess LIBERO didn't have this issue since they trained all their transformers from scratch?
I'm currently debugging an issue with FSDP and offloading sharded model weights. For some reason, even if I put a cuda synchronize + torch garbage collection + gc, on specific clusters it seems to maintain the memory on the gpu. Let me know if anyone else has experienced this!
I've tried manually getting rid of potentially remaining gradients, blocking all threads until garbage collection executes. Yet the issue still seems to persist
TIL: A robotics dataset with no-op actions removed is different from acting out non no-op actions from the same starting state due to drift and momentum/velocity of the robot
Trying to figure out a way to reverse engineer an SFTed model and identify what trajectories it was trained on. I have the dataset, but seems like the probability of the best few trajectories happening in the dataset are similar. Any insights would be appreciated!
Lots of work has gone into LLMs in 2025, with a shift towards the later half going into agents and applications in robotics like Tesla Optimus, Claude Code, GPT Codex. Maybe we'll see more usage of RL.
Let me know what else I should add!
This paper caught my eye today: arxiv.org/pdf/2512.19428
It claims we don't need attention heads, just Grassmann flows.
I want to share my intuition:
1. What are Grassmann flows?
2. What does attention really do?
3. Why COULD this work (or not)?
3.
Smoothly navigating the album itself may be more efficient. No random flips. But attention picks pages based on token content which flows alone might miss.