通过检测推理过程稳定性，实现大模型推理的高效节能。
Efficiently Scaling LLM Reasoning with Certaindex
- 引入Certaindex度量中间答案稳定性，判断何时可提前终止计算。
- 在真实负载中实现50%算力节省与3.3倍吞吐提升，准确率不变。
- 适合需要高效率推理的生产级大模型服务系统使用。
测试时推理算法如思维链、自一致性及蒙特卡洛树搜索虽能提升大模型问题求解能力，但常浪费大量计算生成无效文本。我们观察到这些算法存在答案稳定现象：中间结果在某点后不再变化，继续计算也无法改变最终答案。为此，我们提出Certaindex——一种算法无关的稳定性度量指标，可识别该稳定点，信号计算可提前终止。Certaindex轻量且支持早退、动态分词、群调度等优化，集成至我们设计的推理感知型服务系统Dynasor后，在真实负载中实现最高50%算力节省和3.3倍吞吐提升，准确率无下降。代码已开源。
原文摘要 · Abstract (English)
Test-time reasoning algorithms such as chain-of-thought, self-consistency, and MCTS enhance LLM problem-solving but can wastefully generate many tokens without improving accuracy. At the same time, we observe that these algorithms exhibit answer stabilization: their intermediate solutions often cease to change after a certain point, and further investment of compute does not change their final answer. To quantify this phenomenon, we introduce Certaindex, an algorithm-agnostic metric measuring this evolving stability, signaling when further computation is unlikely to alter the final result. Certaindex is lightweight, can accelerate reasoning program inference via early exit, and further enables dynamic token allocation, gang scheduling, and many opportunities when integrated with real-world LLM serving systems. To quantify real-world benefits, we built Certaindex as a scheduler into Dynasor, our reasoning-aware LLM serving system, and demonstrate up to 50% compute savings and 3.3x higher throughput in real workloads with no accuracy drop. Our code is available at https://github.com/hao-ai-lab/Dynasor.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。