提出首个真正原地计算的实数快速傅里叶变换框架，显著降低训练内存开销。
Memory-Efficient Training with In-Place FFT Implementation
- 利用蝶形运算对称性和频域共轭特性，设计隐式复数编码方案。
- 在多个自然语言任务上实现训练内存大幅下降，最高节省40%以上。
- 适合资源受限场景下的轻量级模型频域优化，尤其利于部署与推理。
快速傅里叶变换（FFT）广泛用于降低深度学习中的内存和计算成本。然而，现有实现（包括标准FFT和实数FFT，rFFT）无法实现真正的原地计算。特别地，rFFT将大小为n的输入映射为大小为n/2+1的复数输出，导致维度不匹配，需额外内存分配。本文提出首个实数域完全原地的FFT框架（rdFFT），保持输入输出内存空间一致性。通过利用蝶形运算的对称性及频域共轭性质，设计隐式复数编码机制，彻底消除中间缓存使用。在多个自然语言理解任务上的实验表明，该方法有效降低训练内存开销，为频域轻量化适配提供了有前景的方向。
原文摘要 · Abstract (English)
Fast Fourier Transforms (FFT) are widely used to reduce memory and computational costs in deep learning. However, existing implementations, including standard FFT and real FFT (rFFT), cannot achieve true in-place computation. In particular, rFFT maps an input of size n to a complex output of size n/2+1, causing dimensional mismatch and requiring additional memory allocation. We propose the first real-domain, fully in-place FFT framework (rdFFT) that preserves input-output memory space consistency. By leveraging butterfly operation symmetry and conjugate properties in the frequency domain, we design an implicit complex encoding scheme that eliminates intermediate cache usage entirely. Experiments on multiple natural language understanding tasks demonstrate the method effectiveness in reducing training memory cost, offering a promising direction for frequency-domain lightweight adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。