给大模型生成内容找依据，判断它是不是真有根据。
ContextCite: Attributing Model Generation to Context
- 用可插拔方法定位生成内容对应的上下文片段
- 能验证回答真实性、优化输入上下文、检测恶意攻击
- 适合关注模型可信性与安全性的研究人员
语言模型如何利用上下文信息生成回复？我们能否判断某句话是基于上下文、误读还是虚构的？为此，我们提出了上下文归因问题：精准定位导致模型生成特定语句的上下文部分（如果有）。接着，我们提出 ContextCite，一种简单且可扩展的上下文归因方法，可应用于任何现有语言模型。最后，我们通过三个应用展示了其价值：(1) 验证生成内容的真实性；(2) 通过剪枝上下文提升回复质量；(3) 检测上下文投毒攻击。代码已开源：https://github.com/MadryLab/context-cite。
原文摘要 · Abstract (English)
How do language models use information provided as context when generating a response? Can we infer whether a particular generated statement is actually grounded in the context, a misinterpretation, or fabricated? To help answer these questions, we introduce the problem of context attribution: pinpointing the parts of the context (if any) that led a model to generate a particular statement. We then present ContextCite, a simple and scalable method for context attribution that can be applied on top of any existing language model. Finally, we showcase the utility of ContextCite through three applications: (1) helping verify generated statements (2) improving response quality by pruning the context and (3) detecting poisoning attacks. We provide code for ContextCite at https://github.com/MadryLab/context-cite.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。