提出无需梯度的参数选择方法，显著降低微调内存与时间开销。
GD-FPS: Growth-Driven Feedforward Parameter Selection for Efficient Fine-Tuning
- 基于前向传播计算激活增长，选择最优参数子集
- 在26个视觉任务上性能媲美或优于主流方法
- 相比现有方法内存减少18倍，速度提升2.7倍以上
参数高效微调（PEFT）已成为适配大规模预训练模型至下游任务的关键策略，但现有方法存在明显局限。基于添加的方法（如Adapters）引入推理延迟和工程复杂性，而基于选择的方法（如梯度驱动参数选择，GPS）需完整反向传播。依赖梯度不仅导致巨大内存占用和显著计算延迟，还使选择结果易受随机批次采样的影响。为此，我们提出生长驱动前向参数选择（GD-FPS）。该方法完全通过前向传播实现，严格无梯度，通过将内在权重幅度按其相对于预训练锚点的相对激活增长进行缩放，以识别最优参数子集。在涵盖图像分类与语义分割的26个视觉任务上，GD-FPS性能达到或超过当前最优的PEFT基线。关键在于，相比GPS，其在参数选择阶段峰值内存降低近18倍，执行速度提升超2.7倍。通过保证确定性选择，GD-FPS为微调提供了高效、快速且鲁棒的解决方案。
原文摘要 · Abstract (English)
Parameter-Efficient Fine-Tuning (PEFT) has emerged as a key strategy for adapting large-scale pre-trained models to downstream tasks, but existing approaches face notable limitations. Addition-based methods, such as Adapters, introduce inference latency and engineering complexity, whereas selection-based methods like Gradient-based Parameter Selection (GPS) require a full backward pass. The reliance on gradients not only incurs massive memory usage and substantial computational latency, but also leaves the selection vulnerable to the randomness of stochastic batch sampling. To resolve this, we propose Growth-Driven Feedforward Parameter Selection (GD-FPS). Operating entirely via forward passes, this strictly gradient-free method identifies the optimal parameter subset by scaling intrinsic weight magnitudes by their relative activation growth against a pre-training anchor. Evaluated on $26$ visual tasks spanning image classification and semantic segmentation, GD-FPS achieves competitive or superior performance over state-of-the-art PEFT baselines. Crucially, compared to GPS, it reduces peak memory usage by nearly $18\times$ and accelerates execution by over $2.7\times$ during the parameter selection stage. By guaranteeing deterministic selection, GD-FPS offers a memory-efficient, fast, and robust solution for fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。