比较大模型训练数据相似性与影响度，发现前者可高效替代后者。
Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior
- 用排名重合度量化数据相似性与影响度的关联性。
- 两者显著相关，但相似性顶部文档在影响度中更稳定。
- 利用不对称性实现低成本高精度的输出溯源方法。
理解大模型行为的一种方式是追溯其输出来源。常用两种度量方法：数据相似性和数据影响度。前者成本低，后者被认为更准确。尽管已有研究在真实任务中对比二者，但尚未应用于输出溯源场景。本文填补这一空白，精确量化两者的共性与差异。通过分别对训练数据按两种度量排序，并计算排名重合度，发现两者显著一致，但存在不对称性：数据相似性排名靠前的文档，在数据影响度中具有更稳定的排名，反之则不然。该现象在OLMo2-1B、Qwen3-1.7B、LlaMa3.2-1B、Gemma3-1B和GPT2等多个模型上均成立。我们利用此不对称性，通过以昂贵的数据影响度优化数据相似性的结果，实现良好的成本-准确性权衡。
原文摘要 · Abstract (English)
One way to understand LLM behavior is to trace its output back to the training data. Two types of measures are commonly used for output tracing: data-similarity and data-influence. The former is cheaper while the latter is believed to be more accurate. Even though many works have compared them for ground-truth tasks, no such comparisons exist for output tracing. Here, we fill this gap and precisely quantify the commonalities and differences between the two measures. We do this by first ranking the training documents according to each measure and then computing the overlap between the two rankings. Our main finding is that the two rankings agree significantly, but there is an asymmetry between them: The top documents of data-similarity are assigned more consistent ranks by data-influence than the other way around. This result is valid across a range of experiments involving OLMo2-1B, Qwen3-1.7B, LlaMa3.2-1B, Gemma3-1B, and GPT2. We exploit the asymmetry to obtain a favorable cost-accuracy trade-off by using the costly data-influence to refine the results of data-similarity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。