Transformer在噪声数据中学习复杂函数时易被简单解困住
Trapped by simplicity: When Transformers fail to learn from noisy features
- 用噪声特征训练Transformer，测试其对无噪输入的泛化能力
- 对随机k-junta任务，噪声下模型准确率显著下降
- 添加敏感度惩罚项可帮助模型摆脱简单解陷阱
大规模语言模型训练数据普遍存在噪声，但其能否正确泛化到无噪输入尚不明确。本文研究Transformer在噪声鲁棒学习中的表现：在含噪声特征的数据上训练后，能否正确预测无噪特征的标签？实验表明，Transformer在部分k-稀疏奇偶性和多数函数上表现良好，而LSTM即使在轻微噪声下也失败。然而，对于随机k-junta任务，尤其当最优解的布尔敏感度低于目标函数时，Transformer通常失败。原因在于其对简单函数的偏好，而最优解敏感度更低。通过构造陷阱解验证该假设，并引入敏感度惩罚损失项，使Transformer能跳出困境。总体而言，噪声存在时Transformer学习布尔函数的能力受限。
原文摘要 · Abstract (English)
Noise is ubiquitous in data used to train large language models, but it is not well understood whether these models are able to correctly generalize to inputs generated without noise. Here, we study noise-robust learning: are transformers trained on data with noisy features able to find a target function that correctly predicts labels for noiseless features? We show that transformers succeed at noise-robust learning for a selection of $k$-sparse parity and majority functions, compared to LSTMs which fail at this task for even modest feature noise. However, we find that transformers typically fail at noise-robust learning of random $k$-juntas, especially when the boolean sensitivity of the optimal solution is smaller than that of the target function. We argue that this failure is due to a combination of two factors: transformers' bias toward simpler functions, combined with an observation that the optimal function for noise-robust learning typically has lower sensitivity than the target function for random boolean functions. We test this hypothesis by exploiting transformers' simplicity bias to trap them in an incorrect solution, but show that transformers can escape this trap by training with an additional loss term penalizing high-sensitivity solutions. Overall, we find that transformers are particularly ineffective for learning boolean functions in the presence of feature noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。