E2E-Fly: An Integrated Training-to-Deployment System for End-to-End Quadrotor Autonomy
A unified sim-to-real pipeline for drone autonomy just cut the gap between training code and flying hardware to near-zero.
The Thesis
E2E-Fly is the first system to wire differentiable physics learning, reinforcement learning, and hardware-in-the-loop validation into a single zero-shot deployment stack for quadrotors. The six real-world tasks demonstrated — trained entirely in simulation, deployed without fine-tuning — suggest the sim-to-real gap for agile drones is an engineering problem, not a fundamental one. If that holds, the timeline for commercial autonomous drone fleets compresses meaningfully.
Catalyst
Differentiable physics simulators have matured enough this year to sit inside end-to-end training loops without prohibitive overhead, and RL tooling (Isaac Lab, Brax) now supports the kind of structured reward design the paper describes. The confluence of fast GPU-accelerated simulation and cheap onboard compute makes hardware-in-the-loop testing at scale newly tractable.
What's New
Prior end-to-end drone control papers — including Agile Flight (UZH), DreamFlight, and Learning to Fly in Seconds — each tackled one slice: training, or transfer, or deployment. E2E-Fly is the first to integrate all three with differentiable physics in a single reproducible pipeline, including explicit latency compensation and noise modeling.
The Counter
The quadrotor sim-to-real problem has been 'nearly solved' repeatedly — Agile Flight papers from Davide Scaramuzza's lab at UZH have shown spectacular acrobatic maneuvers since 2021, and the gap between demo videos and reliable commercial deployment remains vast. E2E-Fly integrates more components in one framework, but integration is not the bottleneck for most drone operators; regulatory certification, edge-case reliability, and unit economics are. A pipeline that trains six tasks in simulation and lands six real flights is impressive academically, but a commercial drone needs to handle unexpected weather, airspace conflicts, and sensor degradation — none of which are addressed here. The cultural momentum score of 0.5 and zero hype hits tell you this is building quietly in robotics labs, not catching fire in product roadmaps. Until there's an open codebase that independent teams can replicate, the 'unified platform' claim is a paper assertion, not a community-verified artifact.
Longs
- NVDA
- AVAV
Shorts
- DJI — proprietary SDK stack becomes a liability if open end-to-end pipelines commoditize flight software
- Traditional autopilot vendors (ArduPilot integrators, Auterion) whose hand-tuned PID stacks get displaced by learned policies
- Drone simulation middleware startups without differentiable physics support
Enablers (Picks & Shovels)
- NVIDIA (Isaac Sim / GPU cluster for parallel RL)
- AMD (Xilinx FPGAs for low-level control interfaces)
- Open-source: Isaac Lab, Brax, ROS2
- ARM (onboard SoCs on flight computers)
Private Watchlist
- Skydio
- Joby Aviation
- Percepto
- Iris Automation
The Paper
Training and transferring learning-based policies for quadrotors from simulation to reality remains challenging due to inefficient visual rendering, physical modeling inaccuracies, unmodeled sensor discrepancies, and the absence of a unified platform integrating differentiable physics learning into end-to-end training. While recent work has demonstrated various end-to-end quadrotor control tasks, few systems provide a systematic, zero-shot transfer pipeline, hindering reproducibility and real-world deployment. To bridge this gap, we introduce E2E-Fly, an integrated framework featuring an agile quadrotor platform coupled with a full-stack training, validation, and deployment workflow. The training framework incorporates a high-performance simulator with support for differentiable physics learning and reinforcement learning, alongside structured reward design tailored to common quadrotor tasks. We further introduce a two-stage validation strategy using sim-to-sim transfer and hardware-in-the-loop testing, and deploy policies onto two physical quadrotor platforms via a dedicated low-level control interface and a comprehensive sim-to-real alignment methodology, encompassing system identification, domain randomization, latency compensation, and noise modeling. To the best of our knowledge, this is the first work to systematically unify differentiable physical learning with training, validation, and real-world deployment for quadrotors. Finally, we demonstrate the effectiveness of our framework for training six end-to-end control tasks and deploy them in the real world.