模型内部激活值能预判推理成败，提升效率
LLMs Encode Their Failures: Predicting Success from Pre-Generation Activations
- 用生成前激活值训练线性探针预测成功率
- 在MATH数据集上推理成本降70%仍胜过单模型
- 模型自有的难易判断与人类不同，适合高效路由
对所有问题进行长序列推理代价高昂，但判断哪些输入真正需要额外计算仍具挑战。我们探究在生成前，是否可从模型内部表示中恢复成功概率，并据此实现更高效的推理。在数学和编码任务上，基于生成前激活值训练的线性探针显著优于题长、TF-IDF等表面特征。利用E2H-AMC数据集（包含相同问题的人类与模型表现），我们发现模型编码了独特的难度认知，且该认知随扩展推理增强，与人类难度感知存在差异。借助这些探针，我们证明通过模型池路由查询，可在保持性能的同时将推理成本降低高达70%，表明即使模型的难易判断偏离人类直觉，内部表示仍能带来实际效率提升。
原文摘要 · Abstract (English)
Running LLMs with extended reasoning on every problem is expensive, but determining which inputs actually require additional compute remains challenging. We investigate whether their own likelihood of success is recoverable from their internal representations before generation, and if this signal can guide more efficient inference. We train linear probes on pre-generation activations to predict policy-specific success on math and coding tasks, substantially outperforming surface features such as question length and TF-IDF. Using E2H-AMC, which provides both human and model performance on identical problems, we show that models encode a model-specific notion of difficulty that is distinct from human difficulty, and that this distinction increases with extended reasoning. Leveraging these probes, we demonstrate that routing queries across a pool of models can exceed the best-performing model whilst reducing inference cost by up to 70\% on MATH, showing that internal representations enable practical efficiency gains even when they diverge from human intuitions about difficulty. Our code is available at: https://github.com/KabakaWilliam/llms_know_difficulty
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。