用图形化方法让张量操作可证明，统一了网络结构与计算流程。
Graphical einops: bridging tensor networks and computation graphs
- 用嵌套的分级管状结构表示张量轴，融合张量网络与计算图优势
- 通过'滑动眼镜'重写规则，将等变性证明简化为图形推导
- 可实现稀疏注意力的高效预处理，适合高效模型设计者
深度学习中的架构图通常仅具表意功能，其隐含的张量程序关系仍需文字和轴操作证明。本文提出一种形式化的图形演算系统，针对einops所依赖的张量编程结构片段，使图表具备证明能力。该系统将张量轴表示为围绕基础类型的嵌套分级管状结构：管壁恢复张量网络中轴的无向性，管内则保留计算图的方向性。核心重写规则为'等级自然性'——可沿管状结构滑动'眼镜'。标准等变性证明因此简化为短小的图形推导。此外，我们展示了该重写系统如何将注意力掩码转换为预处理操作，从而恢复稀疏注意力模块的高效实现。
原文摘要 · Abstract (English)
Architecture diagrams are ubiquitous in deep learning, but they are usually only representational: the tensor-program identities they suggest are still proved by prose and tensor-axis manipulation. We introduce a formal graphical calculus for the structural fragment of tensor programming underlying einops, making such diagrams proof-enabling. Our calculus represents tensor axes as nested graded tubes around a base type. The tube boundary recovers the undirected tensor-network view of axes, while the directed interior retains the operational reading of computation graphs. The key rewrite is grade-naturality: sliding spectacles over tubes. Standard equivariance proofs become short diagrammatic derivations. We additionally demonstrate how our rewrite system may be applied to convert attention masks into pre-processing operations, recovering efficient implementations of sparse attention blocks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。