小模型也能通过测试时学习突破数学难题，开源框架让模型边做题边进化。
ThetaEvolve: Test-time Learning on Open Problems
- 用单个开源大模型+程序库+强化学习，在测试时持续优化解法。
- 在圆堆积和自相关不等式问题上刷新已知最优解，仅用8B参数模型达成。
- 适合想在开放问题中训练模型自我进化的研究者与实践者。
大型语言模型（LLMs）的进展推动了数学发现的突破，如闭源系统AlphaEvolve通过演化程序改进开放问题的界值。然而，该系统依赖前沿模型集成，且为纯推理架构，模型无法内化演化策略。我们提出ThetaEvolve，一个开源框架，简化并扩展了AlphaEvolve，实现测试时上下文学习与强化学习（RL）的高效协同，使模型能从经验中持续学习以优化开放优化问题。ThetaEvolve采用单个LLM、大规模程序数据库以增强探索、批量采样提升吞吐量、懒惰惩罚机制抑制停滞输出，并支持可选奖励塑形以获得稳定训练信号。该框架首次使小型开源模型（如DeepSeek-R1-0528-Qwen3-8B）在圆堆积和首自相关不等式等开放问题上达到新的最优界。在两个模型、四个任务上的实验表明，测试时引入强化学习的ThetaEvolve始终优于仅推理基线，且训练后的检查点展现出更快的进展速度和更优的最终性能，包括对未见过任务的泛化能力。代码已公开：https://github.com/ypwang61/ThetaEvolve
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have enabled breakthroughs in mathematical discovery, exemplified by AlphaEvolve, a closed-source system that evolves programs to improve bounds on open problems. However, it relies on ensembles of frontier LLMs to achieve new bounds and is a pure inference system that models cannot internalize the evolving strategies. We introduce ThetaEvolve, an open-source framework that simplifies and extends AlphaEvolve to efficiently scale both in-context learning and Reinforcement Learning (RL) at test time, allowing models to continually learn from their experiences in improving open optimization problems. ThetaEvolve features a single LLM, a large program database for enhanced exploration, batch sampling for higher throughput, lazy penalties to discourage stagnant outputs, and optional reward shaping for stable training signals, etc. ThetaEvolve is the first evolving framework that enable a small open-source model, like DeepSeek-R1-0528-Qwen3-8B, to achieve new best-known bounds on open problems (circle packing and first auto-correlation inequality) mentioned in AlphaEvolve. Besides, across two models and four open tasks, we find that ThetaEvolve with RL at test-time consistently outperforms inference-only baselines, and the model indeed learns evolving capabilities, as the RL-trained checkpoints demonstrate faster progress and better final performance on both trained target task and other unseen tasks. We release our code publicly: https://github.com/ypwang61/ThetaEvolve
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。