Forge-UGC提升Transformer在异构硬件上的编译与推理效率，兼顾速度与能效。
Forge-UGC: FX optimization and register-graph engine for universal graph compiler
- 分四阶段编译：捕获、优化、降级、调度，支持现代Transformer结构
- 编译速度提升6.9~9.2倍，推理延迟降低18.2%~35.7%，能耗下降30.2%~40.9%
- 提出新评估指标，适合关注NPU编译优化的研究者与开发者
我们提出Forge-UGC（用于通用图编译的FX优化与寄存器图引擎），一种面向异构加速器部署Transformer的四阶段编译器，已在Intel AI Boost NPU上验证。现有框架如OpenVINO和ONNX Runtime常采用不透明编译流程，缺乏可观测性且内存管理薄弱，导致编译成本高、运行时开销大。Forge-UGC通过硬件无关设计，分离图捕获、优化、中间表示降级与后端调度。第一阶段使用torch.export在ATen算子层捕获图，无需手动分解旋转位置编码、分组查询注意力和SwiGLU等现代组件。第二阶段应用六项优化：死代码消除、公共子表达式消除、常量折叠、注意力融合、算子融合与布局优化，使图节点数减少14.2%至21.9%。第三阶段将优化后的图降为带类型注解的中间表示，显式分配虚拟寄存器。第四阶段进行存活分析、线性扫描内存分配，使峰值缓冲区数量减少30%至48%，设备亲和调度减少NPU-CPU切换42%至65%。在涵盖125M至8B参数的六类模型上，基于WikiText-103与GLUE评测，相比OpenVINO和ONNX Runtime，Forge-UGC实现6.9~9.2倍更快编译速度，18.2%~35.7%更低推理延迟，30.2%~40.9%更少每推理能耗。精度保持良好，最大绝对logit差异低于2.1e-5，KL散度低于8.4e-9。此外引入融合增益比、编译效率指数及逐阶段执行剖析，支持系统化评估NPU编译流程。
原文摘要 · Abstract (English)
We present Forge-UGC (FX Optimization and Register-Graph Engine for Universal Graph Compilation), a four-phase compiler for transformer deployment on heterogeneous accelerator hardware, validated on Intel AI Boost NPU. Existing frameworks such as OpenVINO and ONNX Runtime often use opaque compilation pipelines, limited pass-level visibility, and weak buffer management, which can lead to higher compilation cost and runtime overhead. Forge-UGC addresses this with a hardware-agnostic design that separates graph capture, optimization, intermediate representation lowering, and backend scheduling. Phase 1 captures graphs with torch.export at the ATen operator level, supporting modern transformer components such as rotary position embeddings, grouped-query attention, and SwiGLU without manual decomposition. Phase 2 applies six optimization passes: dead code elimination, common subexpression elimination, constant folding, attention fusion, operator fusion, and layout optimization, reducing graph node count by 14.2 to 21.9%. Phase 3 lowers the optimized graph into a typed intermediate representation with explicit virtual register assignments. Phase 4 performs liveness analysis, linear-scan buffer allocation, reducing peak buffer count by 30 to 48%, and device-affinity scheduling, reducing NPU-CPU transitions by 42 to 65%. Across six model families ranging from 125M to 8B parameters, evaluated on WikiText-103 and GLUE, Forge-UGC delivers 6.9 to 9.2x faster compilation than OpenVINO and ONNX Runtime, 18.2 to 35.7% lower inference latency, and 30.2 to 40.9% lower energy per inference. Fidelity is preserved, with max absolute logit differences below 2.1e-5 and KL divergence below 8.4e-9. We also introduce Fusion Gain Ratio, Compilation Efficiency Index, and per-pass execution profiling for systematic evaluation of NPU compilation pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。