OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation

The Thesis

Most self-driving AI stacks face a painful tradeoff: slower models that reason step-by-step in plain language are more accurate, but too slow for real-time use. OneVL proposes a middle path — compress the reasoning into a small bundle of hidden 'latent tokens' that are generated in a single parallel pass rather than one word at a time. The key innovation is that these latent tokens are trained to simultaneously reconstruct language explanations AND predict what the road will look like in the near future, which the authors argue forces the model to internalize genuine causal structure about traffic dynamics rather than just linguistic patterns. On four benchmarks, OneVL is the first latent-reasoning method to beat explicit chain-of-thought accuracy while running at the same speed as a model that gives no reasoning at all. The catch: this is an academic benchmark paper from Xiaomi's embodied intelligence lab, and closed-loop driving performance on real hardware remains undemonstrated.

Catalyst

Vision-Language-Action models — systems that jointly process camera images, language instructions, and output control signals — have matured enough to serve as the backbone for this work, with open pretrained checkpoints making three-stage fine-tuning tractable. Separately, video generation research has produced lightweight token-based world models capable of predicting future frames at low cost, making the visual supervision decoder practical rather than prohibitively expensive to train.

What's New

Earlier latent Chain-of-Thought methods such as COCONUT compressed reasoning into hidden states supervised only by language reconstruction loss, meaning the latent space learned to encode linguistic abstractions of driving rather than the physical dynamics themselves. OneVL adds a second supervisory signal — a visual world model decoder that must predict future-frame tokens from the same latent bundle — forcing the representation to encode road geometry and agent motion trajectories. The authors claim this dual supervision is why their latent approach finally crosses the accuracy threshold that explicit token-by-token reasoning had previously held.

The Counter

The benchmark improvements are real but narrow in scope: every result is on perception-language QA tasks, not on the closed-loop driving metrics that actually determine whether a car stays on the road. Chain-of-thought reasoning in language models has repeatedly shown strong QA scores that fail to transfer to downstream control performance, and there is no evidence here that this pattern is broken. The visual world model decoder is a clever idea, but predicting future frame tokens during training is not the same as learning causal physics — it could just be learning temporal texture correlations that happen to be predictive in the training distribution. The three-stage training pipeline also introduces substantial complexity that may be difficult to reproduce outside a well-resourced lab with access to the exact same pretrained backbone. Finally, the latency advantage is benchmarked against autoregressive token decoding on the same hardware; production autonomous driving stacks use optimized C++ inference pipelines with specialized silicon, and the relative speed benefit of parallel prefilling may shrink considerably in that context.

The Paper