通过并行生成提升大模型属性值提取效率，提速最多13.8倍。
Breaking the Autoregressive Chain: Hyper-Parallel Decoding for Efficient LLM-Based Attribute Value Extraction
- 利用位置编号操控实现无序令牌生成，突破自回归序列限制。
- 单提示词并行解码最多96个令牌，推理时间减少13.8倍。
- 适用于任意独立输出结构任务，适合工业级高效部署。
某些文本生成任务，如属性值提取（AVE），需要从同一文档上下文中解码多个独立序列。标准自回归解码因串行特性速度慢，而输出序列间的独立性为并行处理提供了可能。本文提出超并行解码（Hyper-Parallel Decoding, HPD），通过共享内存与计算加速离线解码。HPD 通过位置 ID 操控实现无序令牌生成，显著提升效率。AVE 实验表明，属性-值对条件独立，可在每个提示中并行生成值。进一步将多个文档叠加至单个提示，可实现每提示最多 96 令牌并行解码。HPD 兼容所有大模型，在不降低输出质量的前提下，将推理成本和总推理时间降低最高 13.8 倍，有望在工业级 AVE 任务中节省数十万美元。尽管针对属性提取设计，但其方法不依赖特定领域假设，理论上可应用于其他具有独立输出结构的场景。
原文摘要 · Abstract (English)
Some text generation tasks, such as Attribute Value Extraction (AVE), require decoding multiple independent sequences from the same document context. While standard autoregressive decoding is slow due to its sequential nature, the independence between output sequences offers an opportunity for parallelism. We present Hyper-Parallel Decoding, a novel decoding algorithm that accelerates offline decoding by leveraging both shared memory and computation across batches. HPD enables out-of-order token generation through position ID manipulation, significantly improving efficiency. Experiments on AVE show that attribute-value pairs are conditionally independent, enabling us to parallelize value generation within each prompt. By further stacking multiple documents within a single prompt, we can decode in parallel up to 96 tokens per prompt. HPD works with all LLMs, and reduces both inference costs and total inference time by up to 13.8X without compromising output quality, potentially saving hundreds of thousands of dollars on industry AVE tasks. Although designed for attribute extraction, HPD makes no assumptions unique to the AVE domain and can in theory be applied to other scenarios with independent output structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。