发现大模型生成中关键分叉词，仅改一个词结果可能天壤之别。
Forking Paths in Neural Text Generation
- 通过分析每一步生成的不确定性，定位影响结果的关键分叉词。
- 在7个任务、4个领域中发现大量分叉词，甚至包括标点符号。
- 无需模型权重或微调，适用于任意模型和数据集，适合安全评估。
估计大语言模型（LLMs）的不确定性对正确评估模型性能及保障用户安全至关重要。然而，以往方法仅关注生成文本的最终答案，忽略了可能显著影响结果的中间步骤。我们假设存在关键分叉标记，即在这些特定标记处重新采样系统，而非其他位置，会导致截然不同的结果。为实证检验此假设，我们提出一种新方法，用于表征文本生成过程中单个标记的不确定性动态，并应用统计模型进行验证。该方法高度灵活，可应用于任意数据集和任意LLM，无需微调或访问模型权重。我们在涵盖4个领域的7个不同任务上分析了LLM响应，发现大量分叉标记，包括令人意外的标点符号，表明LLM往往仅差一个标记就会产生完全不同结果。
原文摘要 · Abstract (English)
Estimating uncertainty in Large Language Models (LLMs) is important for properly evaluating LLMs, and ensuring safety for users. However, prior approaches to uncertainty estimation focus on the final answer in generated text, ignoring intermediate steps that might dramatically impact the outcome. We hypothesize that there exist key forking tokens, such that re-sampling the system at those specific tokens, but not others, leads to very different outcomes. To test this empirically, we develop a novel approach to representing uncertainty dynamics across individual tokens of text generation, and applying statistical models to test our hypothesis. Our approach is highly flexible: it can be applied to any dataset and any LLM, without fine tuning or accessing model weights. We use our method to analyze LLM responses on 7 different tasks across 4 domains, spanning a wide range of typical use cases. We find many examples of forking tokens, including surprising ones such as punctuation marks, suggesting that LLMs are often just a single token away from saying something very different.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。