发现Transformer生成时隐藏的不确定性，可用来判断生成对错。
Uncovering Uncertainty in Transformer Inference
- 通过分析残差流中第n个词嵌入的损失变化轨迹，揭示推理过程逐步优化。
- 损失下降速度越慢，表示生成不确定性越高，错误概率越大。
- 用交叉熵检测不确定性，对成语数据集有良好纠错效果。
我们研究了基于Transformer的语言模型中的迭代推理假设（IIH），旨在理解模型隐层表示如何逐步优化，并观察正确与错误生成之间的差异。实证结果支持了IIH，显示残差流中第n个词嵌入的损失呈现递减趋势。此外，残差嵌入收敛到稳定输出表示的速度反映了生成过程中的不确定性。最后，我们提出一种利用交叉熵检测该不确定性的方法，并在成语数据集上验证其能有效区分正确与错误的词生成。
原文摘要 · Abstract (English)
We explore the Iterative Inference Hypothesis (IIH) within the context of transformer-based language models, aiming to understand how a model's latent representations are progressively refined and whether observable differences are present between correct and incorrect generations. Our findings provide empirical support for the IIH, showing that the nth token embedding in the residual stream follows a trajectory of decreasing loss. Additionally, we observe that the rate at which residual embeddings converge to a stable output representation reflects uncertainty in the token generation process. Finally, we introduce a method utilizing cross-entropy to detect this uncertainty and demonstrate its potential to distinguish between correct and incorrect token generations on a dataset of idioms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。