提出可逆记忆流网络，高效压缩长序列数据。
Invertible Memory Flow Networks
- 用二叉树结构分步合并序列，简化压缩任务
- 实现O(log N)深度和亚线性误差增长，支持长序列处理
- 适合需要高效长序列建模的场景，如视频分析
长序列神经记忆仍是难题。RNN及其变体存在梯度消失问题，Transformer则面临二次计算开销。将长序列压缩为有限固定表示因优化困难而难以实现。本文提出可逆记忆流网络（IMFN），通过分解策略使压缩变得可行：不直接学习端到端压缩，而是将问题拆解为一系列两两合并操作，利用二叉树结构的“扫除”模块完成。每个扫除模块仅学习2→1的简化压缩任务，从而实现O(log N)深度与序列长度的亚线性误差累积。在在线推理中，通过蒸馏得到常数代价的循环学生模型，实现O(1)顺序步骤。实验验证了在长MNIST序列和UCF-101视频上的有效性，展示了对高维长序列数据的压缩能力。
原文摘要 · Abstract (English)
Long sequence neural memory remains a challenging problem. RNNs and their variants suffer from vanishing gradients, and Transformers suffer from quadratic scaling. Furthermore, compressing long sequences into a finite fixed representation remains an intractable problem due to the difficult optimization landscape. Invertible Memory Flow Networks (IMFN) make long sequence compression tractable through factorization: instead of learning end-to-end compression, we decompose the problem into pairwise merges using a binary tree of "sweeper" modules. Rather than learning to compress long sequences, each sweeper learns a much simpler 2-to-1 compression task, achieving O(log N) depth with sublinear error accumulation in sequence length. For online inference, we distilled into a constant-cost recurrent student achieving O(1) sequential steps. Empirical results validate IMFN on long MNIST sequences and UCF-101 videos, demonstrating compression of high-dimensional data over long sequences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。