Illustration brief the model wrote
A robotic arm over a table with a block and a cup. Above the arm, a speech bubble from a language model lists steps like 'pick block', 'pour cup', 'move left'. Two steps have red X marks; the arm reaches for the block. No text or faces.
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Grounds large language model plans in robot affordances, so robot executes only feasible steps and replans upon failure.
What changed
ModelBefore SayCan, large language models could propose high-level plans but had no way to verify if steps were physically possible for a given robot. This paper introduced a system where the LLM proposes candidate actions and a learned affordance model scores each action's feasibility, so the robot only executes actions it can actually perform and re-plans when a step fails.
So what, in plain English
ModelImagine a cook who hears a recipe but only follows steps they have the ingredients and tools for, skipping or adjusting the rest. This lets robots follow natural-language instructions safely by ignoring commands they can't physically carry out.
Contribution to industry
ModelThe paper established the 'LLM reasons, robot filters' pattern now used in industrial and service robotics to turn language commands into executable action sequences. Google and other labs have since integrated similar affordance-filtering loops into pick-and-place, assembly, and mobile manipulation systems.
Typed judgments
Jev★★★★☆ robots · Physical AI 94%
Real hardware 70% · Open code / data 14% · Foundation-model based 95% · Survey / benchmark-only 9%
Probabilities from TypeSafe Jev over the title, venue and abstract; the filters on the atlas apply code-owned thresholds to these numbers.
Editor's notes
Summary. Grounds large language model plans in a robot’s learned affordances so the model proposes steps and the robot scores and executes only the ones it can actually do, replanning when a step fails.
Industry. Established the "LLM reasons, robot filters" pattern industry uses to turn natural-language instructions into safe, executable robot actions on factory and service floors.