Mechatronics engineer by day | Interests: Control, RL, World Models, Computer Vision

London, England
Joined March 2024
Can a robot assemble tight-clearance parts using only its proprioception? No camera, no tactile sensor, no F/T sensor, no joint torque sensing, no priors over the object geometries or poses: just joint position readings. It actually can! Blind policies trained in simulation insert pegs, mesh gears, and thread nuts, and the peg policy works zero-shot in the real world. Here's what we found 🧵 Joint work with @BronarsToni, my wonderful co-first author, who's on the job market right now! Advised by @kushalk_ and @pulkitology.
17
27
11
297
15,562
Abdullah retweeted
The @Tesla Roadster stunts, simulated in MuJoCo! Thrusters are site actuators, the ramp suction uses adhesion actuators, and the builtin box QP decides how hard to fire each jet.
8
19
3
328
15,036
Abdullah retweeted
I've spent years in China building robots. I can tell you this playbook won't work: Pre-seed $: Invent the ultimate actuator. Seed $$: Figure out mass production. The hard part isn't inventing a beautiful topology. It's process engineering. The real delta between the US and China is the ability to execute the enormous amount of physical x mental work required to iterate thousands of times on product designs and manufacturing processes. Partly because much of the tacit knowledge / management systems has left the US, but also because of the sheer human labor required. This isn't something you can solve by buying traditional automation lines either. It requires: 1. Ideation driven by human intuition. Walking up and down the line, talking to assembly workers, and developing hypotheses about what and where improvements can be made. Raw human hours and attention spent understanding a specific manufacturing process. 2. Large batches of experiments across multiple variables. Coordinating experiments that can easily involve 50+ people, issuing instructions, and tracking the execution and results of each experiment. 3. Purpose-built test design to identify key metrics. For example, a change in gear hobbing profile might require accelerated aging tests on a dyno, while a change in bearing tolerance might require NVH testing. 4. Data + human intuition-driven analysis. Interpreting test results and deciding whether to implement a process change or run the next set of experiments. 5. Strict process control. Reissuing work instructions to the line and ensuring every process is followed to the letter. The best manufacturers in Asia, from Foxconn to TSMC, have built their competitive advantage on systematically turning thousands of physical experiments into better manufacturing processes. Their moat isn't just production scale. It's the organizational machinery that makes manufacturing learn and improve faster than anyone else. I don't think most people in SF clock this. Say you're manufacturing actuator gears in-house in the US. After you've made your first article, will you hire a team of 10 process engineers to coordinate the 50 technicians running concurrent experiments with designs, processes, and materials? To understand wear patterns across every pin in a cycloidal drive after 10,000 hours? To reduce backlash from 15 arcmin to 5, then to 1? Just to keep pace with China's progress over the next few years? Do you have the skill set and management systems in place to do this? That's why our approach isn't just to automate manufacturing. It's to automate the process of learning how to manufacture better. Automate simulation and design optimization. Automate assembly and physical testing. Automate the design and execution of process experiments. Feed real-world experimental data back into simulation to continuously improve sim-to-real accuracy. The goal is to turn what currently takes 50 engineers coordinating thousands of experiments into a closed-loop system that continuously learns and improves. Then the same robotic line mass-manufactures the product. PS: Coming back to actuator design, many actuator designs are already commoditized in China. The project in the video is open source, and CNC machining cost in China is just $27 (watch the video). I hope people understand this reality. I'll link more examples in the replies.
54
131
41
1,089
161,722
Abdullah retweeted
Maybe we should have our own "Millennium Problem of Robotics". Robotics academia should be willing to tackle problems that take 5–10 years to solve. Which long-term research problems deserve more attention? My list so far: • Locomanipulation and whole-body coordination • Dexterous, contact-rich manipulation with hands • Force and tactile sensing for perception, learning, and control • Human–robot interaction and collaboration • Evaluation protocols that measure generalization and reliability • Safe, sample-efficient RL in the real world • Long-horizon planning and memory • Lifelong learning and adaptation after deployment • Failure detection, recovery, and knowing when to ask for help • Co-design of robot hardware and learning algorithms What did i miss?
16
17
4
226
11,143
This is a beautiful analysis of the order in which diffusion models resolve uncertainty about information at different abstraction levels in different representation spaces, with significant implications for latent diffusion. panjiashu.github.io/feature-…
9
66
2
545
25,031
millimeter-accurate ground truth, in the environments that give SLAM the least to work with the hilti challenge dataset: 18 sequences on active construction sites with a handheld rig. featureless concrete, scaffolding, buildings that change week to week. surveyed control points, so precise answers exist where texture doesn't find the conditions your pipeline fails on before the field does browser demo, no install: huggingface.co/spaces/harpre… dataset: huggingface.co/datasets/Voxe… #mcap #skills #computervision #ai #artificialintelligence #machinevision #machinelearning #physicalai
1
10
148
6,282
Abdullah retweeted
Can we build world models purely in the latent space, without pixel prediction? We present Contrastive World Models - we train the latent states to maximize mutual information with future observations, without any decoders / pixel reconstruction. Contrastive World Models enables ⚡ Substantially more robust representations 🤖 More efficient training by removing the pixel decoder entirely 🌍 A general approach with minimal assumptions [1/n]
45
172
24
1,907
255,916
Abdullah retweeted
Introducing Real-Time EXPO-FT – Fast and Reliable RL for Real-Time VLA Policies! Real-Time EXPO-FT unlocks π0.5 on challenging dynamic tasks, such as balancing a ball on a plate and striking a ball into the goal (1/6)
6
22
5
204
50,281
Abdullah retweeted
Our work on AthenaZero is the featured story in this month’s issue of @SciRobotics! This bimanual manipulator designed to minimize inertia without compromising control authority. By placing motors closer to the base and using high-torque direct drives, the robot's moving parts weigh about the same as a human arm, making them roughly ten times lighter than typical industrial robots. Learn more about our methodology and watch dynamic, baseball-inspired demos that showcase the robot’s throwing, catching, and batting capabilities on human-comparable timescales: science.org/doi/10.1126/scir…
10
1
76
3,660
Must-read papers of the week ▪️ Programmable World Model ▪️ World in World: Explore the World with World Models ▪️ ActionSplice: In-Flight Action Editing for Interactive World Models ▪️ Earth System World Model for What-If Simulations ▪️ Pelican-Sim 1.0 ▪️ VideoTok4D ▪️ Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model ▪️ Pelican-Sim ▪️ FOCI Policy ▪️ HaWMPO: Hallucination-Aware World Model-based Policy Optimization ▪️ Persimmon v0.1 Model Card ▪️ Does Video Memory Use What It Retrieves? ▪️ IMPLY: Physically Anchored Consistency for World-Model Rollouts ▪️ Negative Self-Distillation: Learning to Reason by Avoiding Flaws Here is the full list of the most interesting papers + links and a weekly AI news digest: turingpost.com/p/what-is-rec…
6
46
6
255
17,610
Abdullah retweeted
Blog for our ECCV paper Demystifying Video Reasoning In 1935, Erwin Schrödinger put a cat in a box with a Geiger counter. Until you open the box, you do not know if the cat is alive or dead. According to Schrödinger, the cat is alive and dead, at the same time. 90 years later, it is the best picture I have for how a video model solves a problem. Link: hokindeng.com/schrodingers-c…
11
65
5,092
NEW The FCC has now added two new categories of devices to our Covered List, which bans new versions from import or sale in America. 1. Advanced robotic devices, such as humanoids and quadrupeds produced in foreign countries. 2. Power inverters produced in foreign countries. This action follows determinations by Executive Branch nat sec agencies that the devices pose unacceptable risks to our national security. It includes exemptions for devices that, based on a DOW or DHS finding, pose no unacceptable threat.
131
483
188
1,896
1,098,967
Abdullah retweeted
I like this paper too, it's actually a very nice to read paper, it flows and is clean
2
17
233
22,066
Abdullah retweeted
“Hilbert Operator for Progressive Encoding” Neural networks hide their knowledge in functions, but most compression methods look at raw weights, which are distorted by scale symmetries and architecture quirks. This DeepMind paper, HOPE, moves compression into Hilbert function space, treating each neuron as a rank-1 operator. It makes pruning, neuron merging, and residual block removal part of the same low-rank projection framework. This gives a data-free, hyperparameter-free way to find which learned functions are redundant, and can easily expose the model’s core representation.
5
60
6
402
24,056
Abdullah retweeted
I found it to be lesser known that the standard definitions of KL estimator in RL losses produce the wrong gradients for expected KL. For example, I stumbled upon the discovery that while the k3 estimator (used in DeepSeek series) is an unbiased estimate for the value of expected reverse KL, its gradient is incorrect for the gradient of KL. It's worse: the gradient is actually an unbiased estimate for the gradient of expected forward KL. So k3 estimator as is defined does the opposite of what it tries to do (p1-2). Then I went deeper into this rabbit hole, after my colleague recommended this paper: arxiv.org/abs/2506.09477. I find two addition things adopted by verl and prob some tech reports that don't make sense: - that the original k1 estimator, which is just the raw log ratio, produces a gradient of an expectation of 0: essentially it is trying to optimizing nothing (p3). - that the sequence-level KL, while its value is correctly the sum of all token-level KL, does not have its gradient as the sum of all token-level KL gradients (p4). I had a great time reading this fascinating paper because it shows how the correct token and sequence KL losses should be defined to yield the right expected gradients. I put my full derivation in a blog because I had fun working out every step: riddlehe.github.io/blog/kl-g… The coolest thing when deriving the gradient-correct sequence-KL estimator is: although the loss term (and the gradient) as is written is computationally heavily, there are one mathematical trick and one computational trick to make the complexity much lower. One is to find an unbiased, lower-variance formulation; the other is to compute the key sum-of-log-ratio term with .cumsum so the whole thing becomes easy. I also find that there is an optional path in verl which does something that kinda of makes sense: there is a 'k3+' estimator that does forward in k3 and backward in k2 (0.5 * log_ratio^2). k2 is biased as an estimator of token-KL value but its gradient checks out to be unbiased as an estimator of token-KL gradient. So this is a way to get the right gradient with backward and the right monitoring signal with forward, using the k2 - k2.detach() + k3.detach() trick.
22
89
8
957
110,839
Fundamentally this is why wheeled robots will lose in many environments (over a 5-10 year timeline). In the end you want a robot that can always stay operational, and there are so many ways a legged robot can adjust and stay stable.
We tied our robot up. One policy walks, jumps, and crawls. Restrain it and it finds its own way back to stable walking from a state we never scripted. Behind it is our founder @RogerXJiang , an ex OpenAI scientist who worked on RLHF, the method behind ChatGPT. We're applying that same idea to robots: train one general policy instead of scripting each move. Follow us for more.
7
7
1
72
14,079
Abdullah retweeted
Your policy doesn't need 7B params. It simply needs dense features. Introducing Patch Policy: pretrained ViT + small transformer beats OpenVLA-OFT with 0.7% of its params, and trains on a 5090. Here it inserts a cable (~2mm tol), and does it again as we unplug mid-rollout. 🧵
11
76
17
671
134,081
Aggregator of open-source datasets -> pi-0.5 was ~40k hours dreamzero only 500 you should have enough to get started with
3
15
1
165
18,422
Introducing InfiniteDiffusion, my independent paper accepted to #SIGGRAPH2026! I have one RTX 3090 Ti. No funding, advisors, or team. By day I'm a new grad SWE at Walmart. The paper has two main contributions: - InfiniteDiffusion: a new approach to infinite generation with diffusion models. - Terrain Diffusion: the world’s first learned procedural terrain generator. Here’s why this matters, and how they are connected. 🧵
153
624
91
6,283
945,980
Announcing Dimensional: the open-source OS for physical space. Command any robot in natural language. Program atoms, not bits.
24
52
9
610
193,712