arXiv:2506.14580cs.CLcs.AI2025-06被引 4

用可执行程序分解生成过程,让大模型回答更可信、可解释。

GenerationPrograms: Fine-grained Attribution with Executable Programs

  • 将生成拆成编程计划与执行两阶段,明确每步操作意图。
  • 在长文本问答和多文档摘要任务中,文档级与句级归因准确率显著提升。
  • 程序可局部修改优化,适合需要高可信度的场景如科研、医疗。

当前大语言模型在源文档条件下的文本生成表现优异,但常无法提供细粒度的归因,影响结果可验证性和可信度。现有归因方法也无法解释模型如何及为何利用源文档生成答案,限制了可解释性。为此,我们提出模块化生成框架GenerationPrograms,受可执行‘代码代理’架构启发。该框架将生成过程分为两个阶段:首先创建由模块化文本操作(如改写、压缩、融合)组成的可执行程序计划,精准匹配查询需求;其次按计划执行操作生成最终输出。实验表明,GenerationPrograms在两项长文本问答任务和一项多文档摘要任务中,显著提升了文档级与句级归因质量。进一步证明其可作为后验归因方法,优于传统技术。此外,生成的可解释程序支持模块级局部优化,从而持续提升整体归因性能。

原文摘要 · Abstract (English)

Recent large language models (LLMs) achieve impressive performance in source-conditioned text generation but often fail to correctly provide fine-grained attributions for their outputs, undermining verifiability and trust. Moreover, existing attribution methods do not explain how and why models leverage the provided source documents to generate their final responses, limiting interpretability. To overcome these challenges, we introduce a modular generation framework, GenerationPrograms, inspired by recent advancements in executable "code agent" architectures. Unlike conventional generation methods that simultaneously generate outputs and attributions or rely on post-hoc attribution, GenerationPrograms decomposes the process into two distinct stages: first, creating an executable program plan composed of modular text operations (such as paraphrasing, compression, and fusion) explicitly tailored to the query, and second, executing these operations following the program's specified instructions to produce the final response. Empirical evaluations demonstrate that GenerationPrograms significantly improves attribution quality at both the document level and sentence level across two long-form question-answering tasks and a multi-document summarization task. We further demonstrate that GenerationPrograms can effectively function as a post-hoc attribution method, outperforming traditional techniques in recovering accurate attributions. In addition, the interpretable programs generated by GenerationPrograms enable localized refinement through modular-level improvements that further enhance overall attribution quality.

可解释生成归因分析程序化推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。