Generated illustration for Teaching Robots to Listen and Think Harder — Paper 09 of 41Seedream 4.0 · GMI Studio

Illustration brief the model wrote

A robot arm reaching toward a cup on a table. Above the robot, a thought bubble shows two sequential icons: a cup and then a door. A human hand points to a different cup on the table, and the thought bubble changes to show that cup icon followed by the door. The scene is simple shapes and no text.

Provenance. Brief written by deepseek-ai/DeepSeek-V4-Flash via GMI Cloud on 2026-09-18 from the paper's title, primary link and the editor's notes; reviewed editorially. Graphic rendered by seedream-4-0-250828 through the GMI Studio request queue (request f6fb6129…), converted to PNG locally. Summaries can be wrong; the linked paper is the source of truth.

Atlas/Physical Intelligence·Physical AI company·2025

Teaching Robots to Listen and Think Harder

Model tl;dr

Two-level system: VLM breaks complex prompts into single steps for π0 low-level policy, trained on synthetic interactions.

vlmπ0hierarchical policysynthetic labelsconversational robottask correction

What changed

Model

Before, robots could not handle multi-stage requests or mid-task corrections interactively. This paper introduced a hierarchical architecture where a VLM acts as a high-level planner that decomposes natural language commands into simple sub-tasks for a low-level policy, enabling real-time corrections and robust execution.

So what, in plain English

Model

Imagine a robot assistant that listens to a complicated instruction like 'set the table and then pour coffee'—a smart manager inside the robot splits it into easy steps the robot can do one at a time. If you interrupt mid-task to say 'use the red cup instead', the manager adjusts the plan on the fly.

Contribution to industry

Model

This paper contributed a concrete architecture and synthetic data pipeline for training conversational robot assistants that can follow multi-stage instructions and accept live corrections. The interaction model is now deployed in service and retail robots, enabling them to handle complex requests from customers.

Typed judgments

Jev

★★★☆☆ robots · Physical AI 90%

Real hardware 60% · Open code / data 7% · Foundation-model based 94% · Survey / benchmark-only 14%

Probabilities from TypeSafe Jev over the title, venue and abstract; the filters on the atlas apply code-owned thresholds to these numbers.

Editor's notes

Summary. Hi Robot is a two-level system: a high-level VLM breaks complex prompts and live corrections into single steps "whispered" to the π0 low-level policy, trained partly on synthetically labelled interactions.

Industry. Laid out the recipe for conversational robot assistants that handle multi-stage requests and mid-task human corrections, the interaction model service and retail robots now ship with.

Read the paper ↗Official research index ↗All 8 Physical Intelligence papers →