通过对比3D场景扰动前后推理结果，减少3D智能体的幻觉错误。
3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding
- 在3D场景图中注入语义与几何扰动，构建扭曲环境
- 对比原始与扭曲场景下的预测，抑制脱离真实环境的生成
- 无需重训练，适用于真实3D智能体，提升决策可靠性
大型多模态模型越来越多地被用作在3D环境中运行的具身智能体的推理核心，但它们仍容易产生幻觉，导致不安全且无依据的决策。现有的推理时幻觉缓解方法主要针对2D视觉-语言场景，无法迁移到具身3D推理中，因为3D中的错误源于物体存在性、空间布局和几何定位，而非像素级不一致。我们提出3D-VCD，首个面向具身3D智能体的推理时视觉对比解码框架。3D-VCD通过在以物体为中心的表示上施加语义和几何扰动（如类别替换、坐标或尺度篡改），构建扭曲的3D场景图。通过对比原始与扭曲3D上下文下的预测结果，该方法抑制对真实场景证据不敏感的词元，这些词元更可能由语言先验驱动。我们在3D-POPE和HEAL基准上评估3D-VCD，结果表明其在不进行任何重训练的情况下持续提升接地推理能力，确立了基于结构化3D表示的推理时对比解码是一种有效且实用的可靠具身智能路径。
原文摘要 · Abstract (English)
Large multimodal models are increasingly used as the reasoning core of embodied agents operating in 3D environments, yet they remain prone to hallucinations that can produce unsafe and ungrounded decisions. Existing inference-time hallucination mitigation methods largely target 2D vision-language settings and do not transfer to embodied 3D reasoning, where failures arise from object presence, spatial layout, and geometric grounding rather than pixel-level inconsistencies. We introduce 3D-VCD, the first inference-time visual contrastive decoding framework for hallucination mitigation in 3D embodied agents. 3D-VCD constructs a distorted 3D scene graph by applying semantic and geometric perturbations to object-centric representations, such as category substitutions and coordinate or extent corruption. By contrasting predictions under the original and distorted 3D contexts, our method suppresses tokens that are insensitive to grounded scene evidence and are therefore likely driven by language priors. We evaluate 3D-VCD on the 3D-POPE and HEAL benchmarks and show that it consistently improves grounded reasoning without any retraining, establishing inference-time contrastive decoding over structured 3D representations as an effective and practical route to more reliable embodied intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。