让Transformer通过自我纠错迭代提升长序列推理能力
Teaching by Failure: Counter-Example-Driven Curricula for Transformer Self-Improvement
- 用模型自身错误生成训练数据，自动构建纠错课程
- 在复杂任务上实现30倍长度外推性能提升
- 无需人工设计难度，适合提升模型泛化能力的研究者
Transformer模型在处理比训练时更长或结构更复杂的输入时常表现脆弱。我们提出反例驱动课程（CEDC），一种自动化框架，通过持续聚焦模型自身失败来提升鲁棒性。每一步中，CEDC利用当前模型生成多样候选问题，通过快速可执行验证器识别错误预测（反例），并在此基础上微调模型。我们在一系列算法与自然语言任务上评估，包括整数加法、排序、Dyck-2语言识别及三个文本分类基准。相比静态训练和标准课程学习基线，CEDC实现最高30倍的长度外推能力提升，计算效率达均匀数据增强的3.75倍，且无需手动设计难度启发式。我们对反例的详细分析表明，课程能自然适应逐步深入复杂错误模式。研究证实，验证器引导的失败驱动学习是一种简单、高效且强大的提升Transformer泛化能力的新范式。
原文摘要 · Abstract (English)
Transformer models often exhibit brittle extrapolation, failing on inputs that are longer or structurally more complex than those seen during training. We introduce Counter-Example-Driven Curricula (CEDC), an automated framework that improves model robustness by iteratively focusing on its own failures. At each step, CEDC uses the current model to generate a diverse set of candidate problems, employs a fast, executable verifier to identify incorrect predictions (counter-examples), and then fine-tunes the model on a dataset enriched with these discovered failures. We evaluate CEDC on a suite of algorithmic and natural language tasks, including integer addition, sorting, Dyck-2 language recognition, and three text classification benchmarks. Compared to static training and standard curriculum learning baselines, CEDC achieves up to 30x greater length extrapolation, is 3.75x more computationally efficient than uniform data augmentation, and requires no manual difficulty heuristics. We provide a detailed analysis of the counter-examples, showing how the curriculum naturally adapts to target progressively more complex error modes. Our findings establish verifier-guided, failure-driven learning as a simple, powerful, and efficient paradigm for enhancing the generalization capabilities of Transformer models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。