提出离线具身验证基准，测试智能体如何通过多视角检查确认细粒度目标物体
PInVerify: An Offline Embodied Benchmark for Active Instance Verification
- 设计主动实例验证任务，让智能体自主选择视角判断物体是否匹配描述
- 构建包含3000个场景的基准，覆盖18类物体，含误导性视角与不可达区域
- 基于4B/8B参数模型的微调方案达到85.6%准确率，验证了细粒度语义理解可行性
具身智能体在导航至目标物体方面取得显著进展，但到达附近并不保证找到正确实例：细微属性差异（如“白色花卉”与“白色条纹”）通常需要近距离、多视角检查。为此，我们提出主动实例验证（AIV）任务，即智能体围绕候选物体主动选择视角，判断其是否匹配细粒度自然语言描述。将AIV形式化为有限时域决策过程，并引入PInVerify——一个离线具身验证基准：涵盖18类物体的3,000个评估场景，以六扇区导航拓扑提供多视角图像，包含可通行但无信息量的陷阱视角和不可达区域。作为基线，我们构建了无需训练的流水线和基于开源多模态大模型（MLLMs）在设备端（≤8B参数）微调的端到端智能体，采用属性分解、可见性加权多视角追踪器及三种下一最佳视角（NBV）策略。在Qwen3-VL（4B/8B）、SenseNova-SI-1.2-InternVL3-8B、CLIP和SigLIP2上的评估显示，最优MLLM基线比最优嵌入基线提升4.9个百分点；真实框（GT-box）消融实验表明检测差距达+3.1个百分点；在测试的NBV策略中未观察到主动视角选择的稳定增益。经LoRA微调的智能体（SFT+GSPO）达到85.6%准确率。PInVerify旨在推动具身AI中主动、细粒度语义验证的研究。
原文摘要 · Abstract (English)
Embodied agents have made strong progress in navigating to target objects, but reaching the goal vicinity does not guarantee that the agent has found the correct instance: subtle attribute differences (e.g., "white floral" vs. "white striped") often require close-range, multi-view inspection. We address this gap with Active Instance Verification (AIV), a task in which an agent actively selects viewpoints around a candidate object to decide whether it matches a fine-grained natural-language description. We formalize AIV as a finite-horizon decision process and introduce PInVerify, an offline embodied benchmark for AIV: 3,000 evaluation episodes across 18 object categories, delivered as multi-view captures with a 6-sector navigation topology that exposes trap views (navigable but uninformative) and unreachable sectors. As reference baselines we build a training-free pipeline and a LoRA-fine-tuned end-to-end agent around open-source multimodal large language models (MLLMs) at on-device scale ($\leq$8B parameters), with attribute decomposition, a visibility-weighted multi-view tracker, and three next-best-view (NBV) strategies. In our evaluation across Qwen3-VL (4B/8B), SenseNova-SI-1.2-InternVL3-8B, CLIP, and SigLIP2, the best MLLM-based baseline exceeds the best embedding baseline by 4.9 pp; GT-box ablations show a +3.1 pp detection gap; and we do not observe reliable gains from active viewpoint selection within the tested NBV strategies. A LoRA-fine-tuned agent (SFT+GSPO) reaches 85.6%. PInVerify aims to support further work on active, fine-grained semantic verification in embodied AI. Code: https://github.com/Avalon-S/PInVerify.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。