提出语言残留分类法，帮自动驾驶高效使用语言模型。
Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving
- 按推理时语言使用方式分四类：仅训练用、隐式推理、条件调用、每帧生成
- 实测显示条件调用在延迟和算力上最优，兼顾性能与效率
- 适合关注模型轻量化与部署的自动驾驶研发人员
视觉-语言-动作（VLA）模型通过语言统一感知、推理与控制，实现语义对齐、可解释决策和长尾泛化，正重塑自动驾驶。但语言模型在车载端成本高昂：延迟与内存预算紧张，自回归解码具固有串行性。本文将核心问题重定义为‘推理时何时何地使用语言’，因推理开销随每帧部署重复发生，而训练开销仅一次。提出‘语言残留’分类法，按推理阶段的语言使用方式将方法分为四类：仅训练监督（L1）、隐式非文本推理（L2）、条件调用（L3）、全帧生成（L4）。综述代表性方法，并在nuScenes、NAVSIM、Bench2Drive等开环与闭环基准上，从延迟、参数量、内存、浮点运算量、生成词元数五个维度评估其部署特性。进一步梳理自然语言处理与大语言模型中的高效技术如何被适配至自动驾驶领域，揭示其适应约束与驱动动机。持续更新的代码库将开源于GitHub。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models are reshaping autonomous driving (AD) by unifying perception, reasoning, and control through language, enabling semantic grounding, interpretable decisions, and better long-tail generalization. But language is expensive onboard: latency and memory budgets are tight, and autoregressive decoding is inherently sequential. This work reframes the central question as when and where language should act at inference, since inference cost recurs at every deployed frame while training cost is paid once. We introduce the Language Residue taxonomy to organize methods by their inference-time use of language: train-time-only supervision (L1), latent non-textual reasoning (L2), conditional invocation (L3), and full per-frame generation (L4). We review representative methods and tag each across five deployment axes (latency, parameters, memory, FLOPs, tokens), analyzing them on major open- and closed-loop driving benchmarks (e.g., nuScenes, NAVSIM, Bench2Drive). We further trace how efficient methods from NLP/LLM are adapted in AD, identifying the constraints and motivations driving these adaptations. A continuously updated repository will be available at Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。