arXiv:2603.10335cs.CV2026-03

提前预测大模型推理长度,提升效率与准确率。

Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models

  • 通过隐含的'燃料'信号预估思维链长度。
  • 在GPQA-Diamond上误差减半,内存分配频率降低13.37倍。
  • 适合优化多模态模型推理的系统设计者与开发者。

大型多模态模型(LMMs)已成为众多应用的首选。然而,这些模型依赖于长度不可预测的思维链(CoT)过程,常导致计算资源利用低效(因内存碎片化)和准确性不佳(因思考不足或过度)。我们实证发现,CoT过程具有极简形式,其行为独立于具体生成样本,暗示可通过一个表示“推理可用资源”的隐藏参数预先估计CoT长度。基于此,我们提出Fuel Gauge,首个提取该隐藏信号并提前预测CoT长度的方法。我们在两个下游任务中验证其有效性:预测性KV缓存分配,缓解多模态服务中的内存碎片;以及CoT长度调节,缓解思考不足与过度。在文本、图文及视频-文本问答基准上的大量实验表明,Fuel Gauge具有效果显著、泛化性强且实用价值高。例如,在GPQA-Diamond基准上,其CoT长度预测误差低于基线的一半,内存分配频率降低13.37倍。

原文摘要 · Abstract (English)

Reasoning Large Multi-modality Models (LMMs) have become the de facto choice for many applications. However, these models rely on a Chain-of-Thought (CoT) process that is lengthy and unpredictable at runtime, often resulting in inefficient use of computational resources (due to memory fragmentation) and sub-optimal accuracy (due to under- and over-thinking). We observe empirically that the CoT process follows a very simple form, whose behavior is independent of the specific generated samples. This suggests that the CoT length can be estimated ahead of time based on a hidden parameter representing the amount of "fuel" available to support the reasoning process. Based on this insight, we propose Fuel Gauge, the first method which extracts this hidden signal and predicts CoT length ahead of time. We demonstrate the utility on the Fuel Gauge on two downstream tasks: predictive KV cache allocation, which addresses memory fragmentation in LMM serving systems, and CoT length modulation, which mitigates under-thinking and over-thinking. Extensive experiments on LMMs across text-only, image-text, and video-text question answering benchmarks demonstrate the effectiveness, generalizability, and practical value of our Fuel Gauge. For example, on the GPQA-Diamond benchmark, our Fuel Gauge achieves less than half the CoT length prediction error compared to the baseline; this translates into a 13.37x reduction in the memory allocation frequency.

多模态模型思维链预测资源优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。