让大模型通过重复任务变快推理，节省56%算力
Can Past Experience Accelerate LLM Reasoning?
- 用自适应算力分配和记忆机制实现推理加速
- 相同任务重复训练后，算力消耗最多降56%
- 适合想提升推理效率的AI研发者
增加大语言模型（LLMs）推理计算量通常可提升效果，但会延长推理时间。与之不同，人类在经验积累后能更快更好完成任务。本文探究：大模型能否通过反复接触相关任务而加快推理速度？为此，我们系统性地定义了任务相关性和算力预算计算两个维度的问题设置。提出SpeedupLLM框架，基于自适应算力分配和记忆机制，理论上保障推理加速行为的实现与评估。在多种问题相似度、记忆方法和推理方法下进行综合实验，结果表明，大模型确实可通过过往经验实现更快推理，在合适记忆与推理方法下，算力成本最高降低56%。
原文摘要 · Abstract (English)
Allocating more compute to large language models (LLMs) reasoning has generally been demonstrated to improve their effectiveness, but also results in increased inference time. In contrast, humans can perform tasks faster and better with increased experience and exposure. Hence, this paper aims to investigate the question: Can LLMs also become faster at reasoning through recurrent exposure on relevant tasks, and if so, how can it be achieved? To address these questions, we first formalize the problem setting of LLM reasoning speedup systematically in the dimensions of task relevancy and compute budget calculation. We then propose SpeedupLLM, a theoretically guaranteed framework to implement and benchmark such reasoning speedup behaviour based on adaptive compute allocation and memory mechanisms. We further conduct comprehensive experiments to benchmark such behaviour across different question similarity levels, memory methods, and reasoning methods. Results show that LLMs can generally reason faster with past experience, achieving up to a 56% reduction in compute cost when equipped with appropriate memory and reasoning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。