让大模型在推理时自动判断何时停止,既省算力又保准确。
LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning
- 基于模型生成中的语气词捕捉推理线索,用轻量探针判断是否可提前退出。
- 在GSM8K上减少40%-65%的token消耗,同时保持或提升准确率。
- 只需一次训练即可跨任务、跨温度使用,支持用户自定义置信度控制。
大型推理模型通过生成长链思维在复杂任务上表现优异,但常出现‘过度思考’现象:在已有足够信息时仍持续推理,浪费计算资源且可能降低准确率。现有早停方法或依赖额外采样策略、辅助验证模型,或仅作为事后分析,缺乏理论保证。本文提出LYNX,一种在线早停机制,将模型自身隐藏状态的感知转化为可控置信度的停止决策。该方法在生成中自然出现的推理线索(如“嗯”“等等”)处附加退出判断,利用强制退出的监督信号,在这些线索位置训练轻量级探针,并通过分叉置信预测封装得分,实现无需分布假设的早停控制。关键的是,探针仅需在通用数学语料上训练一次,即可无须调整地应用于不同基准、解码温度乃至非数学任务。在三个模型家族(1.5B至32B参数)上,每个基模型仅一个数学训练探针即实现优异的准确率-效率权衡:在GSM8K上减少40%-65%的token,准确率不降或提升;在MATH-500上准确率最高提升12点,节省约35%-60%的token;在AIME 2024上恢复基线准确率,节省超50%的token;在非数学任务CommonsenseQA上零样本迁移,准确率小幅提升,最多节省70%的token。相比现有最优早停方法,LYNX提供更优或相当的帕累托前沿,全程在线运行,无需推理时代理模型,且提供显式、可调的置信度保障。
原文摘要 · Abstract (English)
Large reasoning models achieve strong performance on complex tasks by generating extended chains of thought, but they often "overthink": continuing to reason long after they have enough information to answer correctly. This wastes inference-time compute and can hurt accuracy. Existing attempts to stop early either manipulate decoding with extra sampling and heuristics, rely on auxiliary verifier models, or operate only as post-hoc analysis pipelines without formal guarantees. We introduce LYNX, an online early-exit mechanism that turns a model's own hidden-state awareness into confidence-controlled stopping decisions. LYNX attaches exit decisions to naturally occurring reasoning cues (e.g., "hmm", "wait") during generation, trains a lightweight probe on hidden states at those cue tokens using supervision from forced exits, and wraps the resulting scores in split conformal prediction to obtain distribution-free control over premature exits. Crucially, we train and calibrate this probe once on a generic mathematical corpus and reuse it unchanged across benchmarks, decoding temperatures, and even non-mathematical tasks. Across three model families spanning 1.5B to 32B parameters, a single mathematically trained probe per base model yields strong accuracy--efficiency tradeoffs. On GSM8K, LYNX matches or improves baseline accuracy while reducing tokens by 40--65\%; on MATH-500 it improves accuracy by up to 12 points with roughly 35--60\% fewer tokens; on AIME 2024 it recovers baseline accuracy with more than 50\% token savings; and on CommonsenseQA, a non-math benchmark, it transfers zero-shot with modest accuracy gains and up to 70\% fewer tokens. Compared to state-of-the-art early-exit methods, LYNX offers competitive or superior Pareto frontiers while remaining fully online, requiring no proxy models at inference, and providing explicit, user-tunable confidence guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。