将大模型空间推理能力压缩到小模型，实现高效3D场景问答
Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoT
- 用7B教师模型教2.29B学生模型，通过不确定加权损失蒸馏知识
- 推理延迟降低8.7倍，模型大小减少3倍，仍保持54-72%性能
- 引入隐式思维链（Hidden CoT）提升推理，适合移动端部署
大规模3D视觉语言模型（VLMs）如LLaVA-3D具备强大的空间推理能力，但因计算成本高难以部署。本文提出一种知识蒸馏框架，将7B教师模型的空间推理能力迁移至2.29B学生模型。该方法使推理延迟降低8.7倍，模型尺寸缩小3倍，同时保留教师模型54-72%的性能。框架采用VGGT作为视觉编码器，并设计多任务蒸馏流程，结合不确定性感知的损失加权机制。为在无思维链（CoT）数据下提升推理能力，提出“隐式思维链”（Hidden CoT）：可学习的潜在令牌作为生成答案前的内部草稿板，这是首个在蒸馏3D VLM中使用潜在草稿板的尝试。学生模型联合执行空间描述、深度估计和目标检测。在ScanNet和3D-FRONT数据集上的实验表明，其空间理解能力强，接近68-72%的邻近与接触任务准确率。本框架使资源受限平台上的3D场景问答成为可能。
原文摘要 · Abstract (English)
Large-scale 3D vision-language models (VLMs) like LLaVA-3D offer strong spatial reasoning but are difficult to deploy due to high computational costs. We propose a knowledge distillation framework that transfers spatial reasoning from a 7B teacher to a 2.29B student model. Our approach achieves 8.7x lower inference latency and a 3x reduction in model size while retaining 54-72% of the teacher's performance. The framework utilizes VGGT as the vision encoder and a multi-task distillation pipeline with uncertainty-aware loss weighting. To improve reasoning without chain-of-thought (CoT) data, we introduce "Hidden CoT": learnable latent tokens that serve as an internal scratchpad before answer generation. This is the first use of latent scratchpad reasoning in distilled 3D VLMs. The student model jointly performs spatial description, depth estimation, and object detection. Experiments on ScanNet and 3D-FRONT show strong spatial understanding, reaching 68-72% accuracy on proximity and contact tasks. Our framework enables efficient 3D scene QA on resource-constrained platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。