给大模型的回答找源头，能精准定位知识来自哪篇文档。
DataDignity: Training Data Attribution for Large Language Models
- 用伪造维基百科构建可控数据集，保留真实出处但干扰关键词匹配。
- 新模型ScoringModel让溯源准确率提升至52.2%，在多种攻击场景下仍稳定领先。
- 无需训练的SteerFuse方法也表现优异，适合快速部署或资源受限场景。
审计语言模型输出不仅需判断正确性，还需识别支撑回答的原始文档。本文研究精确定位溯源问题：给定提示、目标模型响应和候选语料库，排序最可能支持该响应的文档。提出FakeWiki，一个包含3,537篇虚构维基风格文章的基准数据集，保留真实出处信息的同时削弱词汇捷径。其包含问答探测、源保持改写、回溯生成变体、难辨伪文档（主题相似但关键事实缺失）及五种查询条件（清洁提示+四种越狱式变换）。评估七种检索基线、一种无训练激活引导融合方法SteerFuse，以及一种监督对比溯源排序器ScoringModel。ScoringModel将响应与文档特征映射到共享空间，使用InfoNCE损失，结合批次内、检索挖掘和反文档负样本进行训练。在九个开源指令微调的大语言模型及五种查询条件下，该模型将平均Recall@10从最强基线的35.0提升至52.2，且无需推理时融合，在41/45的模型-条件组合中胜出。在越狱式变换查询上，平均提升15.7个百分点。结果表明，鲁棒的数据溯源需区分真实支持与主题或词汇相似性。
原文摘要 · Abstract (English)
Auditing language-model outputs often requires more than judging correctness: an auditor may need to identify which source document most likely supports the knowledge expressed in a response. We study this as pinpoint provenance: given a prompt, a target-model response, and a candidate corpus, rank the documents that best support the response. We introduce FakeWiki, a controlled benchmark of 3,537 fabricated Wikipedia-style articles designed to preserve ground-truth provenance while weakening lexical shortcuts. FakeWiki includes QA probes, source-preserving paraphrases, retro-generated variants, hard anti-documents that remain topically similar while removing answer-critical facts, and five query conditions: clean prompting plus four jailbreak-inspired transformations. We evaluate seven retrieval baselines, a training-free activation-steering retrieval-fusion method, SteerFuse, and a supervised contrastive provenance ranker, ScoringModel. ScoringModel maps response and document features into a shared space and is trained with InfoNCE using in-batch, retrieval-mined, and anti-document negatives. Across nine open-weight instruction-tuned LLMs and five query conditions, ScoringModel improves mean Recall@10 from 35.0 for the strongest retrieval baseline to 52.2, without inference-time fusion, and wins 41/45 model-by-condition cells. SteerFuse is usually second-best despite requiring no supervised training, showing that activation-space evidence can efficiently complement text retrieval. On jailbreak-inspired transformed queries, ScoringModel improves Recall@10 by 15.7 points on average over the best baseline. Overall, our work shows that robust training data attribution requires evaluation settings that separate true answer support from topical or lexical resemblance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。