在TensorFlow中实现递归函数的高效静态图支持
Recursive Function Definitions in Static Dataflow Graphs and their Implementation in TensorFlow
- 通过标签机制将递归定义转为静态数据流图
- 支持自动微分，适用于深度学习场景
- 适合需要复杂控制流的现代机器学习系统
现代机器学习系统将计算表示为数据流图。随着神经网络架构日益复杂，对更强大且高效的编程抽象需求迫切。本文提出一种在基于数据流的系统（如TensorFlow）中支持递归函数定义的高效技术。该方法将递归定义转换为包含两个简单但强大的数据流操作的静态数据流图。由于静态图在执行期间不改变，可轻松划分并在分布式和异构环境中高效执行。所提技术充分利用了‘标签’思想，这是自数据流系统诞生以来的核心概念之一。我们证明该技术与自动微分兼容，而后者对聚焦深度学习的应用至关重要。文中描述了该技术在TensorFlow框架中的实际实现原理，并通过实验表明，标签机制对于构建现代数据流系统的高效高层抽象具有决定性作用。
原文摘要 · Abstract (English)
Modern machine learning systems represent their computations as dataflow graphs. The increasingly complex neural network architectures crave for more powerful yet efficient programming abstractions. In this paper we propose an efficient technique for supporting recursive function definitions in dataflow-based systems such as TensorFlow. The proposed approach transforms the given recursive definitions into a static dataflow graph that is enriched with two simple yet powerful dataflow operations. Since static graphs do not change during execution, they can be easily partitioned and executed efficiently in distributed and heterogeneous environments. The proposed technique makes heavy use of the idea of tagging, which was one of the cornerstones of dataflow systems since their inception. We demonstrate that our technique is compatible with the idea of automatic differentiation, a notion that is crucial for dataflow systems that focus on deep learning applications. We describe the principles of an actual implementation of the technique in the TensorFlow framework, and present experimental results that demonstrate that the use of tagging is of paramount importance for developing efficient high-level abstractions for modern dataflow systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。