arXiv:2605.09934cs.CL2026-05被引 2

让多模态工具型模型生成可验证的证据链,提升推理可信度。

TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents

论文配图:TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents
图 1 · 摘自论文原文
  • 生成答案时同步记录每句话的支持工具、证据和逻辑关系
  • 在基准测试中实现95.7%的证据还原准确率,比最强基线高23.8个百分点
  • 适合需要高可信度推理的医疗、金融等严谨场景

多模态大模型通过调用外部工具(如视觉检测、OCR、检索、计算)解决视觉任务,但现有工具型智能体通常只暴露执行轨迹和最终答案,缺乏对每条生成结论的证据来源标注。我们称此缺失为‘证明力缺口’。该缺口使工具使用难以验证与优化,因为有效证据、冗余探索与无支持推理混杂在同一轨迹中。为此,我们提出TRACER框架,实现多模态工具型智能体的可验证生成溯源。它不依赖事后追加引用,而是在生成每个回答句的同时,构建结构化溯源记录,明确标识支持该陈述的工具调用回合、证据单元及语义支持关系(包括引用、压缩、推断三类)。通过模式校验、工具回合对齐、源真实性验证与关系合理性检查,确保溯源记录可信,并转化为可追踪约束与强化学习中的局部信用机制。我们进一步构建了TRACE-Bench,用于从粗粒度多模态工具轨迹中重建句子级溯源。在该基准上,仅增加工具会引入噪声;使用Qwen3-VL-8B模型,TRACER达到78.23%的答案准确率和95.72%的摘要准确率,优于最强闭源基线23.80个百分点。相比仅用工具监督微调,其总测试集工具调用次数由4949次降至3486次。结果表明,可靠的多模态工具推理依赖于对观测的溯源感知,而非单纯增加工具调用次数。

原文摘要 · Abstract (English)

Multimodal large language models increasingly solve vision-centric tasks by calling external tools for visual inspection, OCR, retrieval, calculation, and multi-step reasoning. Current tool-using agents usually expose the executed tool trajectory and the final answer, but they rarely specify which tool observation supports each generated claim. We call this missing claim-level dependency structure the provenance gap. The gap makes tool use hard to verify and hard to optimize, because useful evidence, redundant exploration, and unsupported reasoning are mixed in the same trajectory. We introduce TRACER, a framework for verifiable generative provenance in multimodal tool-using agents. Instead of adding citations after generation, TRACER generates each answer sentence together with a structured provenance record that identifies the supporting tool turn, evidence unit, and semantic support relation. Its relation space contains Quotation, Compression, and Inference, covering direct reuse, faithful condensation, and grounded derivation. TRACER verifies each record through schema checking, tool-turn alignment, source authenticity, and relation rationality, and then converts verified provenance into traceability constraints and provenance-derived local credit for reinforcement learning. We further construct TRACE-Bench, a benchmark for sentence-level provenance reconstruction from coarse multimodal tool trajectories. On TRACE-Bench, simply adding tools often introduces noise. With Qwen3-VL-8B, TRACER reaches 78.23% answer accuracy and 95.72% summary accuracy, outperforming the strongest closed-source tool-augmented baseline by 23.80 percentage points. Compared with tool-only supervised fine-tuning, it also reduces total test-set tool calls from 4949 to 3486. These results show that reliable multimodal tool reasoning depends on provenance-aware use of observations, not on more tool calls alone.

多模态可验证性工具调用溯源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。