通过敏感度丢弃降低大模型训练中的幻觉波动，提升输出可靠性。
Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training
- 提出敏感度丢弃（SenD）方法，固定剔除高波动的嵌入索引以稳定训练。
- 在多个领域将幻觉率降低最高达17%，且不损害下游任务表现。
- 开发高效特征得分（EES）检测器，速度提升2倍，适合大规模训练使用。
随着大语言模型广泛应用，其输出幻觉（事实错误或无关内容）问题日益突出。本研究探究训练动态不确定性与幻觉生成之间的关系，基于Pythia系列模型和多种幻觉检测指标，发现训练过程中存在显著的幻觉波动。为此，提出敏感度丢弃（SenD）训练协议，通过确定性地丢弃具有高变异性的嵌入索引，降低幻觉方差。同时，设计了一种无监督的高效特征得分（EES）检测器，速度为传统EigenScore的2倍。该指标被整合进训练流程，使SenD在计算上可扩展且有效。实验显示，SenD在测试阶段提升Pythia和Meta Llama模型的可靠性最高达17%，并在维基百科、医学、法律和编程等领域的事实准确性上均有改善，同时不影响下游任务性能。
原文摘要 · Abstract (English)
As large language models (LLMs) become increasingly prevalent, concerns about their reliability, particularly due to hallucinations - factually inaccurate or irrelevant outputs - have grown. Our research investigates the relationship between the uncertainty in training dynamics and the emergence of hallucinations. Using models from the Pythia suite and several hallucination detection metrics, we analyze hallucination trends and identify significant variance during training. To address this, we propose Sensitivity Dropout (SenD), a novel training protocol designed to reduce hallucination variance during training by deterministically dropping embedding indices with significant variability. In addition, we develop an unsupervised hallucination detection metric, Efficient EigenScore (EES), which approximates the traditional EigenScore in 2x speed. This metric is integrated into our training protocol, allowing SenD to be both computationally scalable and effective at reducing hallucination variance. SenD improves test-time reliability of Pythia and Meta's Llama models by up to 17% and enhances factual accuracy in Wikipedia, Medical, Legal, and Coding domains without affecting downstream task performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。