发现驾驶模型的规划令牌早期就含关键信息,可大幅精简网络仍保持性能。
Depth-Wise Probing and Pruning of the Planning Token in a Driving Vision-Language-Action Model
- 从32层解码器中逐层提取规划令牌,分析其语义与轨迹兼容性。
- 首层即可准确识别导航指令(97.7%),但轨迹精度仅在最后层达最优(2.11米)。
- 删掉8层后推理速度提升1.33倍,误差增加不足5%,适合部署优化。
视觉-语言-动作(VLA)模型通过深度语言模型做出驾驶决策,但行动所需的具体深度尚不明确。本研究以代表性驾驶VLA为对象,其完整规划由单一规划令牌承载,由生成式规划器解码为轨迹。借助规划器作为轨迹空间的逻辑探针,我们从32个解码器层中逐层解码该规划令牌,并测量两个信号:导航指令的线性可解码性及与冻结原生规划器的轨迹兼容性。诊断结果显示,语义意图在早期即可被线性解码:首层命令探测准确率达97.7%,远高于16.7%的随机水平;而与原生规划器的兼容性则随深度逐步提升,开环平均L2误差在最后一层降至最低值2.11米。首层学习读出可弥补大部分差距,表明规划信息虽早期存在,但尚未以部署规划器期望的形式表达。按规划令牌角度偏差排序并移除8层,在约5%相对开环误差增长内实现1.33倍解码器加速。在当前样本量下,未发现家族特异性退化。结果限于ORION检查点与Bench2Drive设置。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models route driving decisions through a deep language model, but it is unclear how much of that depth the action itself requires. We study a representative driving VLA whose entire plan is carried by a single planning token that a generative planner decodes into a trajectory. Borrowing the planner as a trajectory-space logit lens, we decode the planning token from every one of the 32 decoder layers and measure two signals: the linear decodability of the navigation command and trajectory compatibility with the frozen native planner. Our diagnostic shows that semantic intent is linearly decodable early: command-probe accuracy reaches 97.7\% after the first decoder layer, compared with 16.7\% chance. In contrast, compatibility with the frozen native planner improves gradually across depth, with open-loop Avg-L2 reaching its minimum of 2.11\,m only at the final layer. Learned readouts from the first layer recover much of this gap, indicating that planning information is already present early but is not yet represented in the format expected by the deployed planner. Ranking decoder layers by the angular deviation they induce in the planning token permits removal of 8 of 32 layers within an approximately 5\% relative open-loop error increase and yields a measured 1.33$\times$ decoder speedup. At the evaluated sample size, no family-specific degradation is statistically resolved. These findings are limited to the evaluated ORION checkpoint and Bench2Drive setup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。