用大模型检测论文引用是否靠谱，提升生成内容可信度。
Document Attribution: Examining Citation Relationships using Large Language Models
- 将引用溯源转化为文本蕴含任务，零样本直接判断
- 在AttributionBench上准确率提升2.4%（跨域数据集）
- 发现注意力机制对溯源有效，小模型多层表现优于基线
随着大语言模型（LLMs）越来越多地应用于文档任务，如摘要、问答和信息提取，用户更关注从给定文档中检索信息而非依赖模型自身知识。因此，确保系统的可信赖性与可解释性变得至关重要。核心解决方案是溯源，即将生成结果追溯到原始文档。然而，由于LLMs可能产生不准确或模糊的回答，评估其引用可靠性尤为重要。为此，本文提出两种方法：(1) 一种零样本方法，将溯源问题建模为简单的文本蕴含任务；使用flan-ul2，在AttributionBench的ID和OOD数据集上分别比最佳基线提升0.27%和2.4%。(2) 探索注意力机制对溯源过程的影响；使用较小的flan-t5-small模型，除第4层及第8至11层外，其余各层的F1分数均优于基线。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) are increasingly applied to document-based tasks - such as document summarization, question answering, and information extraction - where user requirements focus on retrieving information from provided documents rather than relying on the model's parametric knowledge, ensuring the trustworthiness and interpretability of these systems has become a critical concern. A central approach to addressing this challenge is attribution, which involves tracing the generated outputs back to their source documents. However, since LLMs can produce inaccurate or imprecise responses, it is crucial to assess the reliability of these citations. To tackle this, our work proposes two techniques. (1) A zero-shot approach that frames attribution as a straightforward textual entailment task. Our method using flan-ul2 demonstrates an improvement of 0.27% and 2.4% over the best baseline of ID and OOD sets of AttributionBench, respectively. (2) We also explore the role of the attention mechanism in enhancing the attribution process. Using a smaller LLM, flan-t5-small, the F1 scores outperform the baseline across almost all layers except layer 4 and layers 8 through 11.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。