arXiv:2605.13647cs.CL2026-05

为复杂大模型工作流提供编译优化,提前生成多精度-延迟平衡配置。

FlowCompile: An Optimizing Compiler for Structured LLM Workflows

论文配图:FlowCompile: An Optimizing Compiler for Structured LLM Workflows
图 1 · 摘自论文原文
  • 编译时全局探索工作流设计空间,构建可复用的配置集。
  • 在多个基准上实现最高6.4倍加速,优于启发式和路由方法。
  • 适合需要灵活部署与动态调整的生产级大模型应用。

结构化大模型工作流通过预定义图结构让专用子代理协同完成复杂任务,已成为强大抽象。然而,如何在模型选择、推理预算和工作流结构组成的组合空间中平衡准确率与延迟,仍是挑战。现有成本感知方法多将优化视为路由问题,在推理时根据训练目标选择配置。我们提出从编译视角优化:在部署前,系统可全局探索设计空间,构建一组覆盖多样准确率-延迟权衡的可复用工作流配置。受机器学习编译器启发,我们提出FlowCompile,一种结构化大模型工作流编译器。它将工作流分解为子代理,对各子代理在多种配置下进行性能采样,并通过结构感知代理组合测量值,估算整体准确率与延迟。单次编译即可识别出多样化高质量配置,无需重训练或在线适应。在多个工作流和挑战性基准上的实验表明,FlowCompile持续优于启发式优化与基于路由的基线,最高实现6.4倍加速。编译生成的配置集作为可复用优化产物,支持运行时根据偏好灵活部署,并助力下游选择或路由。

原文摘要 · Abstract (English)

Structured LLM workflows, where specialized LLM sub-agents execute according to a predefined graph, have become a powerful abstraction for solving complex tasks. Optimizing such workflows, i.e., selecting configurations for each sub-agent to balance accuracy and latency, is challenging due to the combinatorial design space over model choices, reasoning budgets, and workflow structures. Existing cost-aware methods largely treat workflow optimization as a routing problem, selecting a configuration at inference time for each query according to the accuracy-latency objective used during training. We argue that structured LLM workflows can also be optimized from a compilation perspective: before deployment, the system can globally explore the workflow design space and construct a reusable set of workflow-level configurations spanning diverse accuracy-latency trade-offs. Drawing inspiration from machine learning compilers, we introduce FlowCompile, a structured LLM workflow compiler that performs compile-time design space exploration to identify a high-quality, reusable trade-off set. FlowCompile decomposes a workflow into sub-agents, profiles each sub-agent under diverse configurations, and composes these measurements through a structure-aware proxy to estimate workflow-level accuracy and latency. It then identifies diverse high-quality configurations in a single compile-time pass, without retraining or online adaptation. Experiments across diverse workflows and challenging benchmarks show that FlowCompile consistently outperforms heuristically optimized workflow configurations and routing-based baselines, delivering up to 6.4x speedup. The compiled configuration set further serves as a reusable optimization artifact, enabling flexible deployment under varying runtime preferences and supporting downstream selection or routing.

大模型编译工作流优化推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。