让大模型推理更省力：动态预测思考长度，节省26%算力。
Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning
- 用轻量适配器预测思考token数与未来收益关系。
- 比固定预算节省26%算力，保持准确率不变。
- 适合追求高效推理的模型部署与测试场景。
我们提出Re-FORC，一种自适应奖励预测方法，可基于查询预测未来思考token数量对应的预期奖励。Re-FORC在推理模型上训练轻量适配器，证明随着推理长度增加和模型增大，预测效果更优。该方法实现：1）对无潜力推理链提前终止，相比固定预算截断减少最高26%计算量，同时保持准确率；2）优化模型与思考长度选择，性能超越单一最大模型——峰值准确率提升1.7个百分点，达到最大模型准确率所需算力减少最多12%；3）自适应测试时缩放，在最大算力下平均提升9.9个百分点准确率，优于基于置信度的基线。Re-FORC支持通过每token成本阈值动态控制推理长度，并可预先估算计算时间。
原文摘要 · Abstract (English)
We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens. Re-FORC trains a lightweight adapter on reasoning models, demonstrating improved prediction with longer reasoning and larger models. Re-FORC enables: 1) early stopping of unpromising reasoning chains, reducing compute by up to 26\% compared to fixed-budget cutoffs, while maintaining accuracy, 2) optimized model and thinking length selection that outperforms the largest model alone--- reaching 1.7 percentage points higher peak accuracy while needing up to 12\% less compute to match the largest model's accuracy, 3) adaptive test-time scaling, which increases accuracy by 9.9 percentage points (on average at maximum compute) over confidence-based baselines. Re-FORC allows dynamic reasoning with length control via cost-per-token thresholds while estimating computation time upfront.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。