arXiv:2605.18818cs.AIcs.LG2026-05被引 1

构建可量产的文档智能系统,用微服务架构整合OCR与大模型处理流程。

Operationalizing Document AI: A Microservice Architecture for OCR and LLM Pipelines in Production

  • 采用微服务分层设计,分离GPU推理与CPU调度,提升资源利用率。
  • 实测每小时处理数千页多页文档,发现OCR是端到端延迟主要瓶颈。
  • 支持独立水平扩展,适合需要高并发文档处理的生产场景。

学术研究多聚焦新模型,导致模型定义与生产部署间存在巨大鸿沟。为弥合这一差距,我们提出一种微服务架构,集成分类、光学字符识别(OCR)及大语言模型结构化字段提取等多模型流水线,并分享在每小时处理数千页多页文档的实际经验。核心设计包括:混合分类策略、将GPU密集型推理与CPU调度分离、异步处理流水线中大量IO操作,以及独立的水平扩展机制。通过批量分析发现两个意外结论:端到端延迟主要由OCR而非语言模型解析决定;系统性能上限由共享的GPU推理能力决定,而非工作节点数量。本研究旨在为实践者提供可落地的文档理解系统架构模式,真正实现模型的生产级部署。

原文摘要 · Abstract (English)

Academic research tends to focus on new models for document understanding creating a wide gap in the literature between model definition and running models at production scale. To close that gap, we present a microservice architecture that encapsulates pipelines of multiple models for classification, optical character recognition (OCR), and large language model structured field extraction as well as our experience running this pipeline on thousands of multi-page documents per hour. We describe our primary design decisions, including a hybrid classification, separation of GPU-bound inference from CPU-bound orchestration, use of asynchronous processing for the many IO-bound operations in the pipeline, and an independent, horizontal scaling strategy. Using batch profiling, we identified two surprising qualitative findings that shape production deployments: OCR, not language-model parsing, dominates end-to-end latency, and the system saturates at a concurrency determined by shared GPU-inference capacity rather than worker count. Our goal is to provide practitioners with concrete architectural patterns for building document understanding systems that work beyond the benchmark; effectively operationalizing models in production.

文档AI微服务OCRLLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。