用余弦投影实现小参数高效微调，精度高且省资源。
MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection
- 将低秩微调的权重变化投射到余弦空间，分层筛选关键频率。
- 在多任务上均超越现有方法，计算量和内存需求大幅降低。
- 适合资源受限场景下的大模型快速适配，尤其适合嵌入式部署。
我们提出一种新型微调方法 MaCP（Minimal yet Mighty adaptive Cosine Projection），在仅需极少参数与内存的前提下，显著提升大基础模型的性能。其核心思想是利用余弦变换的优异能量集中与去相关特性，将低秩适配中的权重变化投影至离散余弦空间，并按频谱层级划分，保留各层最关键的频率成分。大量实验表明，MaCP 在单模态任务（如自然语言理解、生成、摘要）和多模态任务（如图像分类、视频理解）中均表现卓越，相比现有方法，在保持更高准确率的同时，显著降低计算复杂度与内存占用。
原文摘要 · Abstract (English)
We present a new adaptation method MaCP, Minimal yet Mighty adaptive Cosine Projection, that achieves exceptional performance while requiring minimal parameters and memory for fine-tuning large foundation models. Its general idea is to exploit the superior energy compaction and decorrelation properties of cosine projection to improve both model efficiency and accuracy. Specifically, it projects the weight change from the low-rank adaptation into the discrete cosine space. Then, the weight change is partitioned over different levels of the discrete cosine spectrum, and each partition's most critical frequency components are selected. Extensive experiments demonstrate the effectiveness of MaCP across a wide range of single-modality tasks, including natural language understanding, natural language generation, text summarization, as well as multi-modality tasks such as image classification and video understanding. MaCP consistently delivers superior accuracy, significantly reduced computational complexity, and lower memory requirements compared to existing alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。