让AI根据题目难易自动调节思考深度,既不浪费精力也不遗漏关键步骤。
ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping
- 用窗口熵识别推理关键节点,动态控制思考强度。
- 在多类任务上实现性能超越与推理成本降低40%以上。
- 适合需要高效精准推理的复杂多模态应用开发者。
近期多模态大模型在解决复杂文本与视觉任务方面取得显著进展,但普遍存在对简单问题过度思考、对难题探索不足的问题。为此,我们提出ARES框架,通过任务难度自适应分配推理资源。核心发现:单个词元熵噪声大,但窗口熵(HWE)能可靠捕捉关键推理时刻;降低HWE利于简单题,增加则对难题至关重要。ARES采用两阶段训练:第一阶段构建按难度配比推理长度的多模态数据集,赋予模型初始难度感知能力;第二阶段引入自适应熵策略优化(AEPO),以HWE为探索触发信号,并设计分层熵奖励与动态KL控制决定探索程度。大量实验表明,ARES在数学、逻辑及多模态基准测试中表现更优,推理效率更高,在远低于商业系统成本下缩小了性能差距。
原文摘要 · Abstract (English)
Recent advances in multimodal large reasoning models (MLRMs) have substantially improved their ability to solve complex textual and visual tasks. However, these models tend to overthink on simple problems, producing unnecessarily lengthy reasoning traces, while under-exploring on challenging ones, leading to missed solutions. To address this imbalance, we propose ARES, a unified open-source framework for adaptive reasoning that dynamically allocates exploration effort based on task difficulty. Our approach is motivated by two key empirical findings: (i) while single-token entropy is noisy, high window-entropy (HWE) tokens (token-level entropies averaged under a sliding window) can reliably capture reasoning-critical moments; and (ii) reducing HWE usage benefits easy problems, while increasing it is essential for solving hard ones. Building on these insights, ARES introduces a two-stage training pipeline. In the Adaptive Cold-Start stage, we curate multimodal and textual data paired with reasoning traces of length proportional to problem difficulty, equipping the model with initial difficulty awareness. In the second stage, we develop Adaptive Entropy Policy Optimization (AEPO), which uses HWE tokens as exploration triggers to decide when to explore, and a hierarchical entropy reward with dynamic KL control to decide how much to explore. Extensive experiments demonstrate that ARES achieves superior performance and reasoning efficiency across diverse mathematical, logical, and multimodal benchmarks, while closing the gap to leading commercial systems under significantly lower inference costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。