Generated illustration for Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight — Paper 27 of 41Seedream 4.0 · GMI Studio

Illustration brief the model wrote

A drone with a camera on a gimbal flying through a gate. The drone's body is tilted, camera pointed toward the next gate. Background shows a simple racing course with gates. A thought bubble above the drone shows a simulated (dream) version of the same scene.

Provenance. Brief written by deepseek-ai/DeepSeek-V4-Flash via GMI Cloud on 2026-09-18 from the paper's title, primary link and the editor's notes; reviewed editorially. Graphic rendered by seedream-4-0-250828 through the GMI Studio request queue (request 537eb4f8…), converted to PNG locally. Summaries can be wrong; the linked paper is the source of truth.

Atlas/UZH Robotics and Perception Group·Research lab·2026

Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight

Model tl;dr

Applies DreamerV3 model-based RL to learn vision-based drone racing policies from pixels, reaching 9 m/s on real hardware with emergent gate-tracking.

model-based reinforcement learningdrone racingvision-based controldreamerv3embodied aisim-to-real

What changed

Model

Before this paper, model-free RL methods required large amounts of real-world data or complex sim-to-real transfers for drone racing. This paper shows that model-based RL (DreamerV3) can learn directly from pixels in simulation and transfer to real flight, achieving agile speeds with learned camera orientation behavior.

So what, in plain English

Model

Drones usually need explicit programming or tons of real-world practice to learn to race. This paper shows a drone can learn to fly fast by first dreaming about flying in a simulator, then waking up and doing it for real, even steering its camera to look at gates automatically.

Contribution to industry

Model

Contributed a practical method for learning drone racing policies with moderate compute/data, demonstrating model-based RL as a viable alternative to model-free methods for agile autonomy. Research labs and companies developing autonomous drones for inspection, delivery, or racing can use this approach to train policies efficiently in simulation before deploying on real hardware.

Typed judgments

Jev

★★★☆☆ drones · Physical AI 97%

Real hardware 95% · Open code / data 10% · Foundation-model based 17% · Survey / benchmark-only 6%

Probabilities from TypeSafe Jev over the title, venue and abstract; the filters on the atlas apply code-owned thresholds to these numbers.

Editor's notes

Summary. Uses DreamerV3 model-based RL to learn pixel-to-command drone racing policies, emerging with camera-pointing behaviour that keeps gates in view, and flying real hardware up to 9 m/s.

Industry. Positions model-based RL as a practical route to agile autonomy for teams whose data and compute budgets are far smaller than model-free methods require.

Read the paper ↗Official publication archive ↗All 4 UZH Robotics and Perception Group papers →