用电路发现技术选模块微调，训练快6倍且不加参数。
Circuit Fine-Tuning for Compute-Efficient Transformer Adaptation
- 基于初始化前的电路发现选关键模块微调
- 20轮达峰值精度，训练浮点运算少2.3至6.6倍
- 适合算力有限但需快速适配视觉任务的场景
参数高效微调（PEFT）已成为适配视觉变换器（ViTs）到下游任务的主流方法。尽管参数量常被视为效率指标，但它并不等同于计算效率：参数稀疏方法仍可能每步消耗完整模型的训练成本，且通常需要长时间调度才能达到最佳精度。本文提出计算高效的电路微调（CFT），利用电路发现技术——传统上用于解释已训练模型——在训练前筛选出需微调的模块。我们将其归因机制从训练后的任务头改为近零初始化的探测头，从而隔离骨干网络对目标分布的响应，而非特定分类器偏好。CFT仅微调恢复出的子图，无需学习率预热，平均在约20个周期内达到峰值精度（相比强基线的44–96周期），训练浮点运算减少2.3–6.6倍，墙钟时间最多降低16倍，同时增加零参数和无推理开销。在标准视觉迁移基准（VTAB-1k）、分层骨干（Swin）、域偏移医学影像（CBIS-DDSM）及视觉语言模型（Gemma-3 on CUB-200）上的实验验证了CFT的有效性。代码见 https://github.com/UriKialy/CFT。
原文摘要 · Abstract (English)
Parameter-Efficient Fine-Tuning (PEFT) has become the de facto standard for adapting Vision Transformers (ViTs) to downstream tasks. While parameter count has been the dominant efficiency metric in PEFT, it does not imply \textit{compute efficiency}: parameter-sparse methods can still incur full-model training cost per step, and typically need long schedules to reach peak accuracy. We introduce Circuit Fine-Tuning (CFT), a compute-efficient framework that uses circuit discovery---conventionally used to explain trained models---to select modules for fine-tuning before training. Whereas attribution is conventionally formulated against a trained task head, we formulate it against a near-zero-initialized probe head, which isolates the response of the backbone to the target distribution rather than the preferences of a particular classifier. CFT then fine-tunes only the recovered subgraph. CFT needs no learning-rate warmup and reaches peak accuracy in ${\sim}20$ epochs on average---versus $44$--$96$ for strong PEFT baselines---yielding $2.3$--$6.6\times$ fewer training FLOPs and up to $16\times$ less wall-clock time, while adding zero parameters and no inference operations. Experiments across a standard visual transfer benchmark (VTAB-1k), hierarchical backbones (Swin), domain-shifted medical imaging (CBIS-DDSM), and a vision-language model (Gemma-3 on CUB-200) demonstrate the effectiveness of CFT. Code is available at https://github.com/UriKialy/CFT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。