将神经元级的神经网络认证规范自动编译为张量级实现,提升开发效率与性能。
A Tensor-Based Compiler and a Runtime for Neuron-Level DNN Certifier Specifications
- 基于栈式中间表示和形状分析,自动桥接神经元级设计与张量级执行的语义鸿沟。
- 生成的张量计算具有结构稀疏性,通过g-BCSR格式实现双重压缩,存储效率高。
- 支持快速开发新认证器,性能接近手写优化代码,适合安全关键系统开发者。
深度神经网络(DNN)的不可解释性促使人们采用基于抽象解释的认证方法来建立实际系统中的信任。然而,现有方案仅支持有限种类的认证器,且在不同应用场景中开发或修改认证器仍极为困难。这是因为认证器的数学设计以神经元为单位,而其实际实现和执行则在张量层面进行,导致设计与实现间存在语义断层。为此,我们提出一个编译框架,可自动将神经元级的认证器规格转换为张量级、层级的实现。核心创新包括一种新的基于栈的中间表示(IR)以及一种形状分析技术,用于推断模拟神经元语义所需的隐含张量操作。在提升过程中,形状分析以最小张量尺寸创建对应操作。该IR还支持领域特定优化(如重写规则)。运行时,生成的张量计算表现出与网络架构相关的稀疏性,但这种稀疏性与现有格式不匹配。为此,我们引入g-BCSR——一种双层压缩格式,将张量表示为大小可变的块集合,每个块可内部稀疏。结合本编译器与g-BCSR,可轻松开发新认证器,并在多种DNN上分析其有效性。尽管灵活性高,其性能仍可媲美手工优化实现。
原文摘要 · Abstract (English)
The uninterpretability of DNNs has led to the adoption of abstract interpretation-based certification as a practical means to establish trust in real-world systems that rely on DNNs. However, the current landscape supports only a limited set of certifiers, and developing new ones or modifying existing ones for different applications remains difficult. This is because the mathematical design of certifiers is expressed at the neuron level, while their implementations are optimized and executed at the tensor level. This mismatch creates a semantic gap between design and implementation, making manual bridging both complex and expertise-intensive -- requiring deep knowledge in formal methods, high-performance computing, etc. We propose a compiler framework that automatically translates neuron-level specifications of DNN certifiers into tensor-based, layer-level implementations. This is enabled by two key innovations: a novel stack-based intermediate representation (IR) and a shape analysis that infers the implicit tensor operations needed to simulate the neuron-level semantics. During lifting, the shape analysis creates tensors in the minimal shape required to perform the corresponding operations. The IR also enables domain-specific optimizations as rewrites. At runtime, the resulting tensor computations exhibit sparsity tied to the DNN architecture. This sparsity does not align well with existing formats. To address this, we introduce g-BCSR, a double-compression format that represents tensors as collections of blocks of varying sizes, each possibly internally sparse. Using our compiler and g-BCSR, we make it easy to develop new certifiers and analyze their utility across diverse DNNs. Despite its flexibility, the compiler achieves performance comparable to hand-optimized implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。