ORCA通过多模型协作推理,显著减少视觉语言模型的幻觉和对抗攻击漏洞。
ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models
- 设计循环式推理框架,用多个小模型交叉验证图像理解结果。
- 在POPE数据集上使幻觉问题改善3.64%至40.67%,对抗攻击下平均提升20.11%。
- 无需重训练或防御机制,即可实现稳定纠错与可追溯决策,适合高可靠性场景。
大型视觉语言模型(LVLMs)虽具强大多模态能力,但易受内在错误引发的幻觉及外部攻击影响,限制其在真实场景中的可靠性。本文提出ORCA,一种基于推理时结构化推理的智能体框架,通过调用一组小型视觉模型(参数少于30亿)实现对预训练LVLM的事实准确性和对抗鲁棒性的提升。ORCA采用观察-推理-批判-行动循环,以证据性问题查询多个视觉工具,验证跨模型不一致性,并迭代修正预测,全程无需访问模型内部或重新训练。同时,其保存中间推理轨迹,支持可审计决策。尽管主要针对对象级幻觉,ORCA亦展现出无需对抗训练的涌现式鲁棒性。我们在三类场景评估:(1)干净图像上的幻觉基准;(2)未加防御的对抗扰动图像;(3)应用防御技术的对抗扰动图像。在POPE幻觉基准上,ORCA相较独立LVLM提升3.64%至40.67%。在对抗扰动下的POPE测试中,平均准确率提升20.11%。结合防御技术处理对抗扰动的AMBER图像时,性能提升达1.20%至48.00%。结果表明,ORCA为构建更可靠、鲁棒的多模态系统提供了有效路径。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) exhibit strong multimodal capabilities but remain vulnerable to hallucinations from intrinsic errors and adversarial attacks from external exploitations, limiting their reliability in real-world applications. We present ORCA, an agentic reasoning framework that improves the factual accuracy and adversarial robustness of pretrained LVLMs through inference-time structured inference reasoning with a suite of small vision models (less than 3B parameters). ORCA operates via an Observe-Reason-Critique-Act loop, querying multiple visual tools with evidential questions, validating cross-model inconsistencies, and refining predictions iteratively without access to model internals or retraining. ORCA also stores intermediate reasoning traces, which supports auditable decision-making. Though designed primarily to mitigate object-level hallucinations, ORCA also exhibits emergent adversarial robustness without requiring adversarial training or defense mechanisms. We evaluate ORCA across three settings: (1) clean images on hallucination benchmarks, (2) adversarially perturbed images without defense, and (3) adversarially perturbed images with defense applied. On the POPE hallucination benchmark, ORCA improves standalone LVLMs performance by +3.64% to +40.67% across different subsets. Under adversarial perturbations on POPE, ORCA achieves an average accuracy gain of +20.11% across LVLMs. When combined with defense techniques on adversarially perturbed AMBER images, ORCA further improves standalone LVLM performance, with gains ranging from +1.20% to +48.00% across metrics. These results demonstrate that ORCA offers a promising path toward building more reliable and robust multimodal systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。