用傅里叶特征让小模块高效微调大模型，提升效果还省资源。
Towards Efficient Post-Training via Fourier-Driven Adapter Architectures
- 在轻量适配器中引入随机傅里叶特征，分解表示为高低频成分。
- 在多个基准上性能优于或媲美现有方法，计算开销极低。
- 适合需要高效微调大模型的研究者和部署场景。
我们提出一种名为傅里叶激活适配器（FAA）的新框架，用于高效微调大规模预训练语言模型。通过在轻量级适配器模块中引入随机傅里叶特征，FAA将中间表示分解为互补的低频与高频分量，实现对语义信息的频率感知调制。该设计使模型在微调过程中可选择性强调有信息量的频段，同时保持冻结主干的表征能力。在GLUE、E2E NLG及指令微调基准上的大量实验表明，FAA在保持低计算与内存开销的前提下，持续取得与现有参数高效微调方法相当甚至更优的性能。消融实验进一步验证了频率感知激活与自适应加权机制的有效性，凸显FAA作为后训练大模型高效稳健方法的潜力。
原文摘要 · Abstract (English)
We propose a novel framework, termed Fourier-Activated Adapter (FAA), for parameter-efficient fine-tuning of large pre-trained language models. By incorporating random Fourier features into lightweight adapter modules, FAA decomposes intermediate representations into complementary low- and high-frequency components, enabling frequency-aware modulation of semantic information. This design allows the model to selectively emphasize informative frequency bands during adaptation while preserving the representational capacity of the frozen backbone. Extensive experiments on GLUE, E2E NLG, and instruction-tuning benchmarks demonstrate that FAA consistently achieves competitive or superior performance compared to existing parameter-efficient fine-tuning methods, while maintaining low computational and memory overhead. Ablation studies further verify the effectiveness of frequency-aware activation and adaptive weighting mechanisms, highlighting FAA as a robust and efficient approach for post-training large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。