通过引导生成路径提升大模型推理效率，加速40.5%且不损失准确率。
Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs
- 基于早期答案置信度选择最优生成前缀，动态引导后续分支。
- 在数学、常识和符号推理任务中，推理延迟最高降低40.5%。
- 适合需要高效推理的部署场景，如实时问答与边缘计算。
为提升大语言模型（LLMs）的推理能力，自一致性已成为流行方法，通过多次采样结合多数投票实现。然而，现有方法因需大量采样而计算成本高、耗时长。本文提出路径一致性（path-consistency），利用早期生成答案的置信度识别最具潜力的前缀，并指导后续分支的生成。该方法通过动态引导后续生成，有效缓解了自一致性中随机或低效采样的错误与冗余。实验表明，路径一致性显著降低推理延迟，最高提升达40.5%，同时在数学推理、常识推理和符号推理等多种任务上保持原有任务准确率，大幅减少令牌消耗。
原文摘要 · Abstract (English)
To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, current methods are computationally expensive and time-consuming due to the need for numerous samplings. To address this, this paper introduces path-consistency, which leverages the confidence of earlier-generated answers to identify the most promising prefix and guide the generation of subsequent branches. By dynamically guiding the generation of subsequent branches based on this prefix, path-consistency mitigates both the errors and redundancies from random or less useful sampling in self-consistency. This approach reduces errors and redundancies from random sampling, significantly accelerating inference by minimizing token consumption. Our extensive empirical results demonstrate that path-consistency improves inference latency by up to 40.5\%, while maintaining task accuracy across various tasks, including mathematical reasoning, commonsense reasoning, and symbolic reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。