用大模型缓存思维，让小模型推理更准更省
Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Reasoning
- 大模型作师傅缓存高质量推理，小模型作学徒调用
- 相同预算下整体推理准确率提升7.7%，小模型性能最高增36.6%
- 适合资源受限场景下的高效视觉语言推理应用
视觉语言模型（VLMs）在日益复杂的视觉任务中表现卓越，但选择合适的模型规模需在响应质量与成本间权衡。小型VLM运行成本低，但在MMMU等基准测试上表现仅略优于随机猜测。本文提出缓存思维（Cache-of-Thought, CoT）框架，通过大模型（主）与小模型（学徒）协同推理：大模型的高质量输出被缓存，再通过多模态检索与上下文学习机制，为小模型提供支持。我们在多个主流且具有挑战性的通用推理基准上进行了广泛评估，结果显示，在相同预算下，CoT使整体推理性能最高提升7.7%，并使学徒VLM性能最高提升36.6%。代码已开源：https://github.com/UIUC-MONET/Cache-of-Thoughts。
原文摘要 · Abstract (English)
Vision Language Models (VLMs) have achieved remarkable success in a wide range of vision applications of increasing complexity and scales, yet choosing the right VLM model size involves a trade-off between response quality and cost. While smaller VLMs are cheaper to run, they typically produce responses only marginally better than random guessing on benchmarks such as MMMU. In this paper, we propose Cache of Thought (CoT), a master apprentice framework for collaborative inference between large and small VLMs. CoT manages high quality query results from large VLMs (master) in a cache, which are then selected via a novel multi modal retrieval and in-context learning to aid the performance of small VLMs (apprentice). We extensively evaluate CoT on various widely recognized and challenging general reasoning benchmarks, and show that CoT increases overall reasoning performance by up to 7.7% under the same budget, and specifically boosts the performance of apprentice VLMs by up to 36.6%. Our code is available at https://github.com/UIUC-MONET/Cache-of-Thoughts
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。