大模型内部隐含人类感知的问题难度，且可提升训练表现。
LLMs Encode How Difficult Problems Are
- 用线性探测器在多层和词元位置上提取问题难度信号。
- 人类标注难度可被高度解码（ρ≈0.88），且随模型规模上升。
- 引导模型向更“简单”方向优化能减少幻觉、提高准确率。
大型语言模型表现出矛盾现象：能解决复杂问题，却常在看似简单的问题上失败。我们探究了大模型是否内嵌与人类判断一致的问题难度表征，以及该表征在强化学习微调中的泛化能力。在60个模型上，对数学与编码子集的Easy2HardBench数据集训练线性探测器，评估不同层与词元位置的表现。结果发现，人类标注的难度具有强线性可解性（AMC: ρ≈0.88），并呈现清晰的模型规模缩放效应；而模型自身推断的难度信号较弱，且缩放性能差。沿难度方向调控模型时，向“更简单”表示推进可降低幻觉并提升准确率。在Qwen2.5-Math-1.5B的GRPO训练中，人类难度探测器强度增强，与测试准确率正相关；而模型难度探测器则退化，与性能负相关。这表明人类标注提供了稳定的难度信号，强化学习会放大该信号，而基于模型表现的自动难度估计在模型优化过程中逐渐失准。代码与评估脚本已公开以支持复现。
原文摘要 · Abstract (English)
Large language models exhibit a puzzling inconsistency: they solve complex problems yet frequently fail on seemingly simpler ones. We investigate whether LLMs internally encode problem difficulty in a way that aligns with human judgment, and whether this representation tracks generalization during reinforcement learning post-training. We train linear probes across layers and token positions on 60 models, evaluating on mathematical and coding subsets of Easy2HardBench. We find that human-labeled difficulty is strongly linearly decodable (AMC: $ρ\approx 0.88$) and exhibits clear model-size scaling, whereas LLM-derived difficulty is substantially weaker and scales poorly. Steering along the difficulty direction reveals that pushing models toward "easier" representations reduces hallucination and improves accuracy. During GRPO training on Qwen2.5-Math-1.5B, the human-difficulty probe strengthens and positively correlates with test accuracy across training steps, while the LLM-difficulty probe degrades and negatively correlates with performance. These results suggest that human annotations provide a stable difficulty signal that RL amplifies, while automated difficulty estimates derived from model performance become misaligned precisely as models improve. We release probe code and evaluation scripts to facilitate replication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。