用频率切换实现高效多任务学习，仅653万参数提升性能。
Frequency Switching Mechanism for Parameter-E!cient Multi-Task Learning
- 通过正弦调制动态生成任务专用权重，实现零成本参数调整。
- 在密集预测任务上比单任务微调最高提升5.39%，仅需653万可训练参数。
- 适合追求模型轻量化与多任务并行的AI研发人员使用。
多任务学习旨在让单一模型高效解决多个任务，但现有参数高效微调（PEFT）方法大多局限于单任务适应。本文提出一种名为Free Sinewich的参数高效多任务学习框架，通过频率切换实现近零成本的权重调制。具体而言，Sine-AWB（Sinewich）层将低秩因子与卷积先验融合为单一卷积核，并通过正弦变换逐元素调制以生成任务专属权重。引入轻量级时钟网络（Clock Net）生成有界频率，稳定训练过程。理论上，正弦调制提升了低秩适配器的秩，频率分离则降低不同任务权重间的相关性。在密集预测基准测试中，Free Sinewich实现了顶尖的性能-效率平衡（例如，在仅653万可训练参数下，相比单任务微调最高提升5.39%），提供了一种基于频率共享的紧凑且可扩展的多任务学习范式。
原文摘要 · Abstract (English)
Multi-task learning (MTL) aims to enable a single model to solve multiple tasks efficiently; however, current parameter-efficient fine-tuning (PEFT) methods remain largely limited to single-task adaptation. We introduce \textbf{Free Sinewich}, a parameter-efficient multi-task learning framework that enables near-zero-cost weight modulation via frequency switching (\textbf{Free}). Specifically, a \textbf{Sine-AWB (Sinewich)} layer combines low-rank factors and convolutional priors into a single kernel, which is then modulated elementwise by a sinusoidal transformation to produce task-specialized weights. A lightweight Clock Net is introduced to produce bounded frequencies that stabilize this modulation during training. Theoretically, sine modulation enhances the rank of low-rank adapters, while frequency separation decorrelates the weights of different tasks. On dense prediction benchmarks, Free Sinewich achieves state-of-the-art performance-efficiency trade-offs (e.g., up to +5.39\% improvement over single-task fine-tuning with only 6.53M trainable parameters), offering a compact and scalable paradigm based on frequency-based parameter sharing. Project page: \href{https://casperliuliuliu.github.io/projects/Free-Sinewich/}{https://casperliuliuliu.github.io/projects/Free-Sinewich}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。