提出并行反卷积反向传播算法，加速归一化流采样。
Parallel Backpropagation for Inverse of a Convolution with Application to Normalizing Flows
- 用并行算法将反卷积反向传播时间从O(n³)降到O(√n)
- 在标准数据集上采样速度显著提升，比特/维度相近
- 适合需要高效采样的生成模型研究者
反卷积的逆运算在归一化流、图像去模糊等任务中至关重要。传统基于高斯消元的反向传播算法时间复杂度为O(n³)，其中n为图像像素数。本文提出一种快速并行反向传播算法，对正方形图像时间复杂度降至O(√n)，并实现了GPU版本。通常反卷积用于归一化流的采样阶段，导致速度慢。本文将其移至前向传播（图像到隐变量），使采样阶段仅需卷积操作，实现高效采样。利用该并行算法优化反卷积层，获得快速训练。在多种归一化流骨干网络中实现，形成Inverse-Flow模型。在标准数据集上测试，相比先前模型采样时间大幅降低，比特/维度表现相当。
原文摘要 · Abstract (English)
The inverse of an invertible convolution is an important operation that comes up in Normalizing Flows, Image Deblurring, etc. The naive algorithm for backpropagation of this operation using Gaussian elimination has running time $O(n^3)$ where $n$ is the number of pixels in the image. We give a fast parallel backpropagation algorithm with running time $O(\sqrt{n})$ for a square image and provide a GPU implementation of the same. Inverse of Convolutions are usually used in Normalizing Flows in the sampling pass, making them slow. We propose to use the Inverse of Convolutions in the forward (image to latent vector) pass of the Normalizing flow. Since the sampling pass is the inverse of the forward pass, it will use convolutions only, resulting in efficient sampling times. We use our parallel backpropagation algorithm to optimize the inverse of the convolution layer, resulting in fast training times. We implement this approach in various Normalizing Flow backbones, resulting in our Inverse-Flow models. We benchmark Inverse-Flow on standard datasets and show significantly improved sampling times with similar bits per dimension compared to previous models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。