让大模型学会主动遗忘，解决过度思考反而变差的问题。
Free(): Learning to Forget in Malloc-Only Reasoning Models
- 引入可插拔的遗忘模块，动态清理无用推理信息。
- 在所有模型规模上提升性能，长任务中准确率从0%恢复至50%。
- 适合需要长期推理的复杂任务，尤其对超大规模模型有显著帮助。
推理模型通过扩展推理时计算资源来提升问题求解能力，但面临一个关键悖论：过多的推理步骤反而会降低性能。我们将其归因于架构缺陷——标准大语言模型像‘仅分配内存’的引擎，持续积累有效与冗余信息而缺乏清除机制。为此，我们提出Free()LM，通过一个即插即用的LoRA适配器——Free-Module，赋予模型内在自遗忘能力。该模型在推理与清理模式间迭代切换，动态识别并剔除无用上下文块，保持紧凑且无噪声的状态。大量实验表明，Free()LM在8B至685B各规模模型上均实现稳定提升，平均超越顶尖推理基线3.3%，并在IMOanswerBench上以DeepSeek V3.2-Speciale达成新SOTA。尤为关键的是，在长时序任务中，标准Qwen3-235B-A22B模型准确率跌至0%，而Free()LM成功恢复至50%。研究提示：可持续智能不仅需要思考能力，更需遗忘的自由。
原文摘要 · Abstract (English)
Reasoning models enhance problem-solving by scaling test-time compute, yet they face a critical paradox: excessive thinking tokens often degrade performance rather than improve it. We attribute this to a fundamental architectural flaw: standard LLMs operate as "malloc-only" engines, continuously accumulating valid and redundant steps alike without a mechanism to prune obsolete information. To break this cycle, we propose Free()LM, a model that introduces an intrinsic self-forgetting capability via the Free-Module, a plug-and-play LoRA adapter. By iteratively switching between reasoning and cleaning modes, Free()LM dynamically identifies and prunes useless context chunks, maintaining a compact and noise-free state. Extensive experiments show that Free()LM provides consistent improvements across all model scales (8B to 685B). It achieves a 3.3% average improvement over top-tier reasoning baselines, even establishing a new SOTA on IMOanswerBench using DeepSeek V3.2-Speciale. Most notably, in long-horizon tasks where the standard Qwen3-235B-A22B model suffers a total collapse (0% accuracy), Free()LM restores performance to 50%. Our findings suggest that sustainable intelligence requires the freedom to forget as much as the power to think.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。