arXiv:2507.01800cs.CVcs.MM2025-07

通过分层聚焦监督,让3D视觉问答模型更像人一样逐步推理。

HCNQA: Enhancing 3D VQA with Hierarchical Concentration Narrowing Supervision

  • 模仿人类由广到窄的搜索过程,分三阶段引导模型聚焦关键区域。
  • 在ScanNet、Matterport3D等数据集上,准确率提升3.2%~5.1%。
  • 适合追求可解释性与稳健推理的3D视觉理解研究者。

3D视觉问答(3D VQA)对于模型理解物理世界和进行空间推理至关重要。当前主流的以答案为中心的训练方法仅监督最终输出,允许模型自由发展推理路径,易导致依赖问题-答案对中的表面模式。此外,虽有慢思考方法提升大语言模型表现,但存在过度简化的问题。为此,本文提出HCNQA,一种基于分层聚焦缩小监督机制的3D VQA模型。该方法模拟人类从宽泛区域逐步聚焦至具体物体的搜索过程,通过分阶段监督关键推理节点,引导模型形成合理有效的推理路径。大量实验表明,该方法能有效确保模型具备理性推理能力,并在ScanNet、Matterport3D等基准上实现3.2%~5.1%的性能提升。代码已公开于https://github.com/JianuoZhu/HCNQA。

原文摘要 · Abstract (English)

3D Visual Question-Answering (3D VQA) is pivotal for models to perceive the physical world and perform spatial reasoning. Answer-centric supervision is a commonly used training method for 3D VQA models. Many models that utilize this strategy have achieved promising results in 3D VQA tasks. However, the answer-centric approach only supervises the final output of models and allows models to develop reasoning pathways freely. The absence of supervision on the reasoning pathway enables the potential for developing superficial shortcuts through common patterns in question-answer pairs. Moreover, although slow-thinking methods advance large language models, they suffer from underthinking. To address these issues, we propose \textbf{HCNQA}, a 3D VQA model leveraging a hierarchical concentration narrowing supervision method. By mimicking the human process of gradually focusing from a broad area to specific objects while searching for answers, our method guides the model to perform three phases of concentration narrowing through hierarchical supervision. By supervising key checkpoints on a general reasoning pathway, our method can ensure the development of a rational and effective reasoning pathway. Extensive experimental results demonstrate that our method can effectively ensure that the model develops a rational reasoning pathway and performs better. The code is available at https://github.com/JianuoZhu/HCNQA.

3D视觉问答推理路径分层监督可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。