模型自生成解法并迭代改进，突破长度与难度泛化瓶颈。
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
- 模型自动生成答案并从中学习，逐步解决更难问题。
- 从10位数加法推广到100位数，性能无饱和迹象。
- 无需改架构或位置编码，适合训练后持续优化的场景。
大语言模型在长度泛化和超出训练分布的复杂问题求解上表现不佳。本文提出一种自改进方法：模型迭代生成并学习自身解法，在保持标准Transformer结构的前提下，逐步攻克更难任务。在算术、字符串操作和迷宫求解等多类任务中，该方法使模型能够解决远超初始训练范围的问题——例如，从10位数加法推广至100位数加法而无性能饱和。部分任务中，筛选正确自生成样本可带来训练轮次间性能的指数级提升。此外，从预训练模型开始能显著加速自改进过程。结果表明，通过受控的弱到强课程，可在不修改位置编码或模型结构的情况下，系统性地教会模型进行逻辑外推。
原文摘要 · Abstract (English)
Large language models often struggle with length generalization and solving complex problem instances beyond their training distribution. We present a self-improvement approach where models iteratively generate and learn from their own solutions, progressively tackling harder problems while maintaining a standard transformer architecture. Across diverse tasks including arithmetic, string manipulation, and maze solving, self-improving enables models to solve problems far beyond their initial training distribution-for instance, generalizing from 10-digit to 100-digit addition without apparent saturation. We observe that in some cases filtering for correct self-generated examples leads to exponential improvements in out-of-distribution performance across training rounds. Additionally, starting from pretrained models significantly accelerates this self-improvement process for several tasks. Our results demonstrate how controlled weak-to-strong curricula can systematically teach a model logical extrapolation without any changes to the positional embeddings, or the model architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。