FuseFlow让稀疏模型在流式硬件上高效运行，支持跨层融合优化。
FuseFlow: A Fusion-Centric Compilation Framework for Sparse Deep Learning on Streaming Dataflow
- 将稀疏模型转为融合的流式计算图，支持跨表达式融合
- 在GPT-3上实现2.7倍加速，证明全融合并非总是最优
- 适合做稀疏模型编译与硬件设计探索的研究者
随着深度学习模型规模扩大，稀疏计算与专用数据流硬件成为提升效率的关键。我们提出FuseFlow，一个将PyTorch中稀疏机器学习模型转换为可重构数据流架构（RDAs）上融合稀疏数据流图的编译器。FuseFlow是首个支持稀疏操作跨表达式融合的编译器，不仅支持核函数间的融合，还集成并行化、数据流顺序和稀疏性分块等优化。它基于周期精确的数据流模拟器，用于分析融合策略的微架构性能。我们在四个真实世界的稀疏模型应用中进行设计空间探索，发现对稀疏模型而言，全融合（端到端模型中所有计算的跨表达式融合）并不总是最优——融合粒度取决于模型特性。FuseFlow还提供启发式方法识别并剪除次优配置。使用FuseFlow，在采用BigBird块稀疏注意力的GPT-3上，相较未融合基线实现了约2.7倍的加速。
原文摘要 · Abstract (English)
As deep learning models scale, sparse computation and specialized dataflow hardware have emerged as powerful solutions to address efficiency. We propose FuseFlow, a compiler that converts sparse machine learning models written in PyTorch to fused sparse dataflow graphs for reconfigurable dataflow architectures (RDAs). FuseFlow is the first compiler to support general cross-expression fusion of sparse operations. In addition to fusion across kernels (expressions), FuseFlow also supports optimizations like parallelization, dataflow ordering, and sparsity blocking. It targets a cycle-accurate dataflow simulator for microarchitectural analysis of fusion strategies. We use FuseFlow for design-space exploration across four real-world machine learning applications with sparsity, showing that full fusion (entire cross-expression fusion across all computation in an end-to-end model) is not always optimal for sparse models-fusion granularity depends on the model itself. FuseFlow also provides a heuristic to identify and prune suboptimal configurations. Using Fuseflow, we achieve performance improvements, including a ~2.7x speedup over an unfused baseline for GPT-3 with BigBird block-sparse attention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。