用多标记预测加速3D场景理解，速度提升同时保持高精度。
Fast SceneScript: Fast and Accurate Language-Based 3D Scene Understanding via Multi-Token Prediction
- 采用多标记预测减少自回归迭代次数，提升推理速度。
- 每轮解码生成最多9个标记，精度不下降，速度显著提升。
- 引入置信度引导解码，适合需要快速准确3D理解的场景。
基于语言模型的感知通用方法在3D场景布局估计和3D物体检测等任务中已达到领先水平，但依赖自回归下一个标记预测，固有速度慢。本文提出Fast SceneScript，一种用于高效精准3D场景理解的结构化语言模型。该方法采用多标记预测（MTP）以减少自回归迭代次数，大幅加速推理。为应对不可靠标记影响精度的问题，引入自推测解码（SSD）并设计置信度引导解码（CGD），改进标记可靠性评分机制。此外，设计参数高效机制，将MTP的参数开销控制在约7.5%。在合成与真实世界基准上的实验表明，Fast SceneScript可在不牺牲精度的前提下，每解码步骤生成最多9个标记。
原文摘要 · Abstract (English)
Recent perception-generalist approaches based on language models have achieved state-of-the-art results across diverse tasks, including 3D scene layout estimation and 3D object detection, via unified architecture and interface. However, these approaches rely on autoregressive next-token prediction, which is inherently slow. In this work, we introduce Fast SceneScript, a novel structured language model for accurate and efficient 3D scene understanding. Our method employs multi-token prediction (MTP) to reduce the number of autoregressive iterations and significantly accelerate inference. While MTP improves speed, unreliable token predictions can significantly reduce accuracy. To filter out unreliable tokens, we adapt self-speculative decoding (SSD) for structured language models and introduce confidence-guided decoding (CGD) with an improved scoring mechanism for token reliability. Furthermore, we design a parameter-efficient mechanism that reduces the parameter overhead of MTP. Extensive experiments on synthetic and real-world benchmarks demonstrate that Fast SceneScript can generate up to 9 tokens per decoder inference step without compromising accuracy, while adding only $\sim7.5\%$ additional parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。