让复杂AI应用的Python代码自动并行，提速近6.5倍
PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications
- 结合编译器与运行时，自动发现调用外部模型的并行机会
- 在真实场景中实现最高6.4倍的端到端加速
- 无需修改代码即可提升性能，适合自动化类应用开发者
复合型AI应用通过通用编程语言（如Python）组合多个机器学习模型调用，广泛应用于软件工程、企业自动化等场景，其端到端延迟成为关键瓶颈。与传统程序不同，这类应用的执行时间主要由外部组件决定，无法通过传统优化系统（如优化编译器）处理。为此，我们提出PopPy系统，可自动挖掘调用重型外部组件的Python应用中的并行性，支持高度表达性的Python子集，且开发者介入极少。PopPy融合提前编译与运行时机制，解决从Python程序中提取并行性的三大挑战：语言复杂性、动态分发和变量突变。在一组真实世界复合型AI应用上，相较于标准Python执行，PopPy实现了最高6.4倍的端到端执行速度提升，同时保持程序的顺序语义不变。
原文摘要 · Abstract (English)
Compound AI applications, which compose calls to ML models using a general-purpose programming language like Python, are widely used for a variety of user-facing tasks, from software engineering to enterprise automation, making their end-to-end latency a critical bottleneck. In contrast to traditional applications, execution time is dominated by the external components, which cannot be handled by traditional language optimization systems, like optimizing compilers. To address this problem, we develop PopPy, a system that can uncover parallelization opportunities in Python applications that invoke these heavy external components, including those used in compound AI applications. PopPy supports a very expressive fragment of Python and requires minimal developer input to uncover parallelism. It combines an ahead-of-time compiler with a runtime, addressing three key challenges in extracting parallelism from Python applications: language complexity, dynamic dispatch, and variable mutation. On a set of real-world compound AI applications, PopPy achieves up to $6.4\times$ speedups in end-to-end execution time compared to standard Python execution while preserving the sequential program semantics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。