根据推理复杂度动态分配计算资源，提升模型效率与准确率
EAGer: Entropy-Aware GEneRation for Adaptive Inference-Time Scaling
- 基于分词熵判断复杂度，仅在高不确定处展开多路径探索
- 在AIME 2025上实现Pass@k提升12%，生成 tokens 减少64%
- 无需训练即可适配不同模型，适合复杂推理场景
随着推理型语言模型和测试时扩展方法的兴起，为同一提示生成多个候选序列常需大量计算。这虽能探索不同推理路径以逼近正确答案，但对所有提示均分配相同算力。基于不同提示具有不同复杂度的假设，我们提出EAGer——一种无需训练的生成方法，通过分词级熵分布感知模型不确定性，减少冗余计算并提升整体性能。EAGer仅在高熵分词处进行多路径分支，并将节省的算力重分配至最需要探索的实例。我们在多个开源模型上验证了EAGer在复杂推理基准上的有效性，在AIME 2025上表现突出。当目标标签可获取（如在RLVR训练流程中），其可实现Pass@k最高+37%、令牌数减少59%；在测试时设置下仍达Pass@k +12%、令牌数减少64%，优于全并行采样。
原文摘要 · Abstract (English)
With the rise of reasoning language models and test-time scaling methods as a paradigm for improving model performance, substantial computation is often required to generate multiple candidate sequences from the same prompt. This enables exploration of different reasoning paths toward the correct solution, however, allocates the same compute budget for each prompt. Grounded on the assumption that different prompts carry different degrees of complexity, and thus different computation needs, we propose EAGer, a training-free generation method that leverages model uncertainty through token-wise entropy distribution to reduce redundant computation and concurrently improve overall performance. EAGer allows branching to multiple reasoning paths only in the presence of high-entropy tokens, and reallocates the saved compute budget to instances where exploration of alternative paths is most needed. We validate EAGer across multiple open-source models on complex reasoning benchmarks, with gains specifically demonstrated on AIME 2025. When target labels are accessible -- as in RLVR training pipelines -- EAGer achieves up to +37% in Pass@k and 59% fewer tokens; in test-time settings it still yields +12% in Pass@k and 64% fewer tokens compared to Full Parallel Sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。