让大模型的内外知识使用更透明可信，生成有据可查的答案。
Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation
- 设计新任务与评估框架，区分内外知识引用
- 提出RAEL框架与INTRALIGN方法，提升引用可信度
- 实验证明检索质量、问题类型影响引用可信性
尽管通过检索增强生成和引用生成可缓解大模型幻觉问题，但其内部知识的使用仍不透明，生成答案的可信度存疑。本文提出上下文优先增强引用生成任务，要求模型在结合外部与内部知识的同时生成可信参考文献，并设计了涵盖答案有用性、引用忠实度与可信度的五项评估指标。我们提出RAEL范式及包含常规数据生成与对齐算法的INTRALIGN方法。实验表明，该方法在跨场景表现上优于基线。扩展实验进一步揭示：检索质量、问题类型与模型知识水平对引用可信度具有显著影响。
原文摘要 · Abstract (English)
While hallucinations of large language models could been alleviated through retrieval-augmented generation and citation generation, how the model utilizes internal knowledge is still opaque, and the trustworthiness of its generated answers remains questionable. In this work, we introduce Context-Prior Augmented Citation Generation task, requiring models to generate citations considering both external and internal knowledge while providing trustworthy references, with 5 evaluation metrics focusing on 3 aspects: answer helpfulness, citation faithfulness, and trustworthiness. We introduce RAEL, the paradigm for our task, and also design INTRALIGN, an integrated method containing customary data generation and an alignment algorithm. Our experimental results show that our method achieves a better cross-scenario performance with regard to other baselines. Our extended experiments further reveal that retrieval quality, question types, and model knowledge have considerable influence on the trustworthiness in citation generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。