通过分阶段处理提升大模型批处理推理效率与稳定性
Cascaded Batch Prompting
- 分两阶段解耦复杂推理与符号定位，缓解批处理不确定性
- 多选题和自然语言推理任务上超越单次提示基线，速度随批次增大而提升
- 适合追求高效稳定推理的部署场景，尤其在资源受限环境下
尽管批处理提示能通过同时处理多个实例提升大语言模型推理效率，但其下游任务性能存在不可预测性。本文提出级联批处理提示（Cascaded Batch Prompting），一种两阶段方法，旨在通过解耦复杂推理与符号定位，解决传统批处理提示的不确定性问题。在多项选择题问答和自然语言蕴含任务上的实验表明，该方法在性能上优于标准单次提示基线，同时推理速度与批大小成正比，已在帕累托前沿确立新基准。
原文摘要 · Abstract (English)
Although batch prompting makes large language model inference more efficient by processing multiple instances simultaneously, it suffers from unpredictable downstream task performance. We propose cascaded batch prompting, a two-stage approach designed to resolve the unpredictability of conventional batch prompting by disentangling complex reasoning from symbol grounding. Experiments on multiple-choice question answering and natural language inference demonstrate that the proposed method outperforms the standard single prompting baseline while achieving a speedup proportional to batch size, establishing a new state of the art on the Pareto frontier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。