@chelseabfinn
Asst Prof of CS & EE @Stanford Co-founder of Physical Intelligence @physical_int PhD from @Berkeley_EECS, EECS BS from @MIT
I'm giving a talk tomorrow at ICML on emergent physical generalization, including π0.7 🤖 3:15 pm @ SCALE workshop in Ballroom 201 scale-icml-2026.github.io
I'm giving a talk on how we can move beyond the scalar reward bottleneck for both robotics & LLMs. ICML RLxF workshop tomorrow at 1:30 pm. x.com/shaohua0116/st…
We’ve been approaching reward supervision for robots the wrong way. I think freeform preferences are part of the answer. A short 🧵
Long term, we need reward models to capture all aspects of performance, like success, outcome quality, and speed. This also includes task-specific axes like: - was the PB spread evenly? - was the apple slightly bruised while bagging it? - was the furniture bumped or scratched?
LLM RL optimizes for sequential reasoning We also optimize over the reasoning strategy, incl parallel trains of thought, aggregation of parallel traces, & sequential reasoning This allows the model to better explore & allocate compute at test time t.co/DkTSllkmvp
Can we translate a rough sense for what to do + VLA prior into successful behavior? Flow reversal steering: 1) runs the VLA's flow ODE backwards to back out noise that's closest to coarse traj 2) runs flow forwards to get closest good behavior Paper: t.co/FUDqDEpeJq
How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper: jadee-dao.github.io/direct/ x.com/_jadelynn/stat…
Can robot foundation models collaborate with themself? We finetune a VLA to be able to control any robot in a team. - matches or outperforms training separate models or a single centralized model for all robots - readily scales to large teams Paper: t.co/yqdSOQ7ead
Scaling RL to long horizons remains a major challenge. Long-horizon Q-learning (LQL) prevents compounding bootstrapping errors by bounding the difference in value over long horizons. It shows large gains over 1-step TD and n-step returns! Paper: arxiv.org/abs/2605.05812 x.com/armaanabraham/…
How can VLAs achieve 95+% reliability? Using RL post-training with EXPO-FT: - π0.5 improves to 30/30 success on all 8 tasks tested - uses only 19 min of RL data on average Paper & videos: pd-perry.github.io/expo-ft/ x.com/perryadong/sta…
EXPO-FT builds on EXPO (arxiv.org/abs/2507.07986) The first idea of EXPO is to train a small Gaussian policy, to edit the VLA's actions. We also continuously distill successful trajectories into the base VLA.
