通过渐进式多标记预测,实现文档解析的五倍加速且精度几乎不变。
P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

- 采用渐进式多标记预测,分阶段提升长距离预测深度。
- 在多个基准上实现最高5倍推理速度提升,准确率损失可忽略。
- 适合需要高吞吐量文档解析的工业级应用,如智能表单处理。
视觉语言模型(VLM)通过端到端图像到结构化文本映射,彻底革新了文档解析,但对密集令牌文档存在显著延迟瓶颈。尽管多标记预测(MTP)已成为加速推理的有前景方法,其在扩展至更深前瞻深度时受限于优化不稳定性。本文提出P-MTP框架,利用轻量级MTP模块与渐进式多标记预测,实现高吞吐量文档解析。我们引入渐进课程损失,基于累积路径可靠性与回溯目标一致性自适应重加权不同前瞻深度,有效抑制长程预测中的梯度噪声,实现从易到难的自动优化过渡。此外,提出置信度门控动态预写机制,在推理中自适应校准推测长度,最大化有效前瞻深度与接受率,减少计算浪费,进一步突破推理加速极限。跨多个基准与架构的实验表明,P-MTP实现了最高5倍的速度提升,且准确率损失微乎其微,首次成功验证了文档解析领域中大规模前瞻MTP的有效性。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have revolutionized document parsing by enabling end-to-end mapping from images to structured text, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emerged as a promising approach for accelerating inference, its potential is constrained by optimization instability when scaling to deeper look-ahead depth. In this paper, we propose \textbf{P-MTP}, a framework that leverages \textbf{Progressive Multi-Token Prediction} with a lightweight MTP module to scale the look-ahead depth for high-throughput document parsing. Specifically, we introduce Progressive Curriculum Loss that adaptively re-weights different look-ahead depths using cumulative path reliability and retrospective target consistency. By effectively suppressing gradient noise in long-range predictions, P-MTP, facilitates an automated easy-to-hard optimization transition, enabling the model to master increasingly distant look-ahead depths. Furthermore, we propose Confidence-Gated Dynamic Drafting to maximize the effective look-ahead depth and acceptance rate by adaptively calibrating speculative length during inference, thereby minimizing computational waste and further pushing the boundaries of inference speedup. Experimental results across multiple benchmarks and architectures demonstrate that P-MTP, achieves up to a $5\times$ speedup with negligible loss in accuracy, providing the first successful validation of extensive look-ahead MTP in the document parsing domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。