为动态张量计算设计流式编程抽象，提升硬件执行效率
Streaming Tensor Programs: A Streaming Abstraction for Dynamic Parallelism
- 引入可变路由与符号化形状，支持动态数据流调度
- 实现动态分块、并行度自适应，延迟降低2.72倍，算力利用率提升2.64倍
- 适合大模型推理中存在数据依赖的动态场景使用
动态行为在张量应用（如机器学习）中日益普遍，许多主流模型包含依赖数据的张量形状和控制流。然而，以往空间数据流加速器（SDA）的编程抽象表达能力有限，迫使动态行为只能静态或低效实现。为此，我们提出流式张量程序（STeP），一种支持动态张量工作负载高效运行在SDA上的流式抽象。STeP引入灵活路由操作符、显式内存层次结构和符号化形状语义，暴露动态数据速率与张量维度。这些能力解锁了动态分块、动态并行化与配置时复用等新优化，使SDA执行能适配动态行为的同时保持数据流效率。在典型LLM层及基于真实轨迹的完整模型上，通过周期级近似模拟验证：动态分块突破了先前工作的帕累托最优边界，动态并行化将延迟降低约2.72倍，配置时复用使算力利用率提升约2.64倍，优于现有SDA抽象及其实现。
原文摘要 · Abstract (English)
Dynamic behaviors are becoming prevalent in tensor applications, like machine learning, where many widely used models contain data-dependent tensor shapes and control flow. However, the limited expressiveness of prior programming abstractions for spatial dataflow accelerators (SDAs) forces these dynamic behaviors to be implemented statically and/or unoptimized. To address these challenges, we present Streaming Tensor Programs (STeP), a streaming abstraction that enables dynamic tensor workloads to run efficiently on SDAs. STeP introduces flexible routing operators, an explicit memory hierarchy, and symbolic-shape semantics that expose dynamic data rates and tensor dimensions. These capabilities unlock new optimizations, like dynamic tiling, dynamic parallelization, and configuration time-multiplexing, that adapt SDA execution to dynamic behaviors while preserving dataflow efficiency. Using a cycle-approximate simulator on representative LLM layers and a full model with real-world traces, STeP enables: dynamic tiling that breaks the Pareto-optimal frontier from prior work, dynamic parallelization that improves latency by ~2.72x, and configuration time-multiplexing that increases compute utilization by ~2.64x over prior SDA abstractions and their implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。