@svlevine
Associate Professor at UC Berkeley Co-founder, Physical Intelligence
How do we run RL with real-time chunking (RTC)? In this work we figured out how to use a small RL policy with a large robot foundation model, where the RL policy observed more recent images (due to faster inference) and steers the policy toward better behaviors! A fun collaboration with Siemens, led by Brian Zhu, Momen Khalil, Emanuele Poggi from Siemens and @ehharrison4 from Berkeley, with lots of amazing contributors!
E Harrison@ehharrison4·Asynchronous VLA inference reduces inference delay, but breaks the Markovian assumption necessary for RL fine-tuning. How can we enable RL fine-tuning of VLAs with async inference? We introduce ARLI: Asynchronous RL with Intermediate Information! …rl-intermediate-information.github.io (1/n)
Seohong wrote a mystery novel. You won't believe whodunit
Seohong Park@seohong_park·Behavioral cloning mystery seohong.me/blog/behaviora… I wrote a new blog post about "mysteries" in behavioral cloning that appear with real-world robot data (e.g., overfitting is "good"). I also tried to demystify them and shared my thoughts!
Thanks Ryan for coming by! This was a fun chat.
Ryan Peterman@ryanlpeterman·Sergey Levine (@svlevine) is one of the world's top robotics researchers and co-founder of Physical Intelligence. We talked about where humanoid robotics is today, thoughts on the Chinese robotics ecosystem, and his predictions for future timelines. In this episode: • Current state of robotics and surprising capabilities so far • Chinese robotics compared to US ecosystem • If OpenAI and Anthropic got into robotics • His top robotics research paper recommendation • Predictions for when humanoid robotics will land Where to watch: • YouTube - t.co/z5y5nJIYyd • Spotify - t.co/R5nOgyet64 • Apple Podcasts - t.co/jOYDGtHtd1 • Transcript - t.co/SivQ80BxAr Thank you to the sponsor of this episode for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at t.co/y8noBzGc3U Chapters: 00:00 Intro 00:37 Where are we today 04:20 Most surprising capabilities so far 07:03 The most inspiring real world robotics 08:36 If OpenAI or Anthropic got into robotics 10:22 Chinese robotics 13:15 Will one lab breakout from the rest 16:59 Thoughts on a concrete roadmap 21:03 Generalization and demonstrating it 26:04 Types of data and which is best for robotics 34:34 Why humanoid robotics differs from Waymo 37:10 If humanoid robotics failed here is why 39:55 Are there hot take modeling architectures in robotics 42:05 Thoughts on AI safety in robotics 46:44 Top robotics research paper recommendation 49:35 Why is Boston Dynamics less top of mind 53:47 Advice for his younger self 56:42 Outro
Latest Deep RL class lectures are now online! youtube.com/playlist?list=… Thanks to @seohong_park, we now have CS185/285 for spring 2026 available to everyone to watch. Course website here: rail.eecs.berkeley.edu/deeprlcourse/ Apologies for a few recording glitches (it's not a perfect system).
Chelsea doing another rock star presentation. Robots can indeed fold laundry and make espresso!
Y Combinator@ycombinator·Robots can already fold laundry, make espresso, clean kitchens, and assemble things. The harder problem is getting them to do those tasks reliably, for long periods of time, without a human babysitting them. At Startup School 2026, @physical_int cofounder @chelseabfinn explains what it takes to build general-purpose robots that work in the real world. She shares how reinforcement learning pushed robot throughput up 2x, how their systems can run autonomously for hours, and why she believes robotics is entering its GPT era: moving from specialized models toward general-purpose systems that can work across tasks, robots, and environments. 00:00 — The State of Physical Intelligence 01:23 — What It Takes to Make Robots Useful 05:11 — The Reliability Problem 07:43 — Reinforcement Learning for Robotics 09:35 — Learning From Failures 12:43 — Training Robots to Improve Themselves 14:21 — Can a Robot Work for 13 Hours Straight? 17:36 — Why Robots Need Memory 21:22 — Building a General-Purpose Robot 25:02 — From Fine-Tuning to Out-of-the-Box Models 27:35 — Training on All the Data 30:20 — One Model That Beats the Specialists 31:21 — Compositional Generalization 37:49 — The GPT Era of Robotics 39:49 — Q&A
Action chunking is a mysteriously effective method. Modern large-scale imitation learning basically doesn't work without it. But why does it actually help? In our new paper, we try to break down the reasons. As the saying goes, what happened next might surprise you... x.com/ajwagenmaker/s…
Learning from suboptimal data is important, because robots make suboptimal data on their own, and the more robots there are, the more data they make. If you want to contribute to building a public, open dataset of suboptimal data, check this out! x.com/zhiyuan_zhou_/…
While we share a lot about our research work at @physical_int, we don't talk much about what it's actually like to work here. Here is a very nice blog post from @Stone_Tao about how working at Pi works, and a little bit about sim at Pi! x.com/Stone_Tao/stat…
@barak1724 I believe there was a Slideslive recording! I assume this means it shows up on the ICML website eventually, I believe they initially make it available only to registered attendees and then eventually release it.
Starting now in ASEM Ballroom 203 after a bit of AV troubles :) see you there! The talk will cover how to build up a general framework for data-driven decision making! x.com/svlevine/statu…
This morning at ICML 2026, I'll be speaking at the Workshop on Decision-Making from Offline Datasets. Come to ASEM Ballroom 203 at 8:15 am KST! I'll talk about foundations of data-driven decision making, from design via model-based optimization to offline RL
If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semantic Action RL! In our paper, @JagdeepBhatia8, @ajwagenmaker, @verityw_ show how RL over VLA prompts enables new tasks and learns blazing fast in the real world!
The idea is simple: instead of running RL over robot actions, run RL over language commands to a VLA. This is much easier, because any vision-language model worth its salt provides a great prior over likely semantic commands, so RL has a much smaller search space.
We can learn a model that provides shaped "process rewards" for robotic RL, that evolves automatically as the policy gets better. This improves performance on benchmarks, and works in the real world! Some fun new work with Raymond Tsao & @ajwagenmaker
In the real world, this leads to drastically faster learning, in just a few thousand time steps.
A new way to do off-policy RL with diffusion: if we have off-policy data, we need to figure out what the diffusion latent steps for it would be with our *current* policy (not the one that collected it), so this requires reversing the diffusion process on off-policy data. x.com/aditya_oberai/…
Flow reversal steering allows "steering" diffusion-based VLAs with high-level actions, for example from VLM reasoning. This also lets us run RL in the diffusion noise space with exploration guided by high-level reasoning: think through a task, then practice it! 👇
Check out the paper here: flow-reversal-steering.github.io A fun collaboration w/ Andy Tang, @verityw_ , @ajwagenmaker, @chelseabfinn
Diffusion (or flow) makes for excellent policies, but training them with RL is notoriously hard: BPTT is unstable, RL over diffusion blows up the horizon. In our new paper, we show how we can optimize flow matching actors by using "one weird trick" -- "approximate" the Jacobian of the flow denoising process with the identity matrix. 👇
Our method (QGF) outperforms using the true Jacobian or BPTT. It is entirely a test-time method (i.e., the policy is trained with BC, the Q-function is trained with TD, and at test-time, optimize the Q-function wrt actions using the identity Jacobian "approximation").
