arXiv:2607.27145cs.CV2026-07

让多模态大模型的空间推理更可信、更省资源,靠的是可读的物体关系提示。

Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications

论文配图:Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications
图 1 · 摘自论文原文
  • 用物体间几何关系生成可读提示,不需训练
  • 在多个基准上提升46%空间判断准确率
  • 轻量版仅40词上下文,适合实时系统

随着多模态大语言模型(MLLMs)在机器人、具身智能和安全监控等关键决策场景中广泛应用,其空间判断的不可解释性限制了操作员信任与可审计性。尽管MLLMs具备强推理能力,但在细粒度空间理解与物体幻觉方面仍存在挑战。先前工作ByDeWay引入了基于分层深度提示(LDP)的无训练框架,通过单目深度估计结构化提示以缓解幻觉。然而,粗粒度深度分层难以解析同一几何平面上的物体间空间关系,如投影关系(“左”“上”)和拓扑关系(“内部”“接触”)。本文提出ByDeWay-V2,将显式的空间关系上下文与深度线索结合,以人类可读谓词形式注入提示,为下游决策提供可审计证据。利用开源开放词汇目标检测器YOLO-World-L,框架计算检测物体间的成对几何关系,并将其作为结构化空间谓词注入MLLM提示,无需任何训练即可连接3D场景深度与2D空间语义。我们在Visual Spatial Reasoning(VSR)与BLINK基准上评估ByDeWay-V2,使用POPE进行幻觉归因分析。在BLINK空间子集上,ByDeWay-V2相较LDP使Qwen2.5-VL的F1相对提升46%,并将BLIP-Base在VSR上的空间推理性能从接近随机恢复至0.53的竞争力水平。最轻配置在CPU上仅需40词上下文,适用于资源受限的实时决策支持场景。

原文摘要 · Abstract (English)

As Multimodal Large Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics, embodied AI, and safety monitoring, the opacity of their spatial judgments limits operator trust and auditability. MLLMs demonstrate strong reasoning but often struggle with fine-grained spatial understanding and object hallucination. Prior work, ByDeWay, introduced Layered-Depth-Based Prompting (LDP), a training-free framework that mitigates hallucinations by structuring prompts using monocular depth estimation. However, coarse depth layering falls short in resolving object-to-object spatial relationships within the same geometric plane, such as projective ("left of", "above") and topological ("inside", "touching") relations. We propose ByDeWay-V2, which integrates explicit spatial relational context alongside depth cues, expressed as human-readable predicates that serve as auditable evidence for downstream decision support. Using an open-vocabulary object detector (YOLO-World-L), our framework computes pairwise geometric relations between detected objects and injects them as structured spatial predicates into the MLLM prompt, bridging 3D scene depth and 2D spatial semantics without any training. We evaluate ByDeWay-V2 on the Visual Spatial Reasoning (VSR) and BLINK benchmarks across multiple MLLMs, with hallucination grounding assessed via POPE. On the BLINK spatial subset, ByDeWay-V2 achieves a 46 percent relative F1 improvement over LDP for Qwen2.5-VL, and recovers BLIP-Base's spatial reasoning on VSR from near-random performance to a competitive F1 of 0.53. Our lightest configuration operates under a strict 40-token context budget on CPU, showing the framework's suitability for resource-constrained, real-time decision-support settings.

空间推理可解释性轻量化多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。