用切比雪夫多项式构建高效微调方法，显著提升模型性能且参数极少。
Chebyshev Manifold Adaptation
- 基于可学习坐标与训练系数的切比雪夫多项式叠加，替代传统线性投影。
- 在CIFAR-10/100和AG News/SST-2上优于全参微调、LoRA等方法。
- 理论证明收敛性，适合资源受限场景下的高效适配。
本文提出一种新的参数高效适配方法ChebyMA（切比雪夫流形适配）。ChebyMA通过在可学习坐标上评估的切比雪夫多项式基的多流形叠加，并由可训练系数矩阵组合，取代标准线性投影，实现高度表达性的连续函数逼近。理论上，我们建立了近似表达性定理，从函数逼近论角度证明单流形ChebyMA在重构误差的Frobenius范数下保证收敛。此外，基于Kolmogorov $n$-宽度直觉，我们展示了多流形叠加（$S > 1$）在解耦高维复杂特征方面的表达优势。在计算机视觉的CIFAR-10、CIFAR-100及自然语言处理的AG News、SST-2数据集上的实验结果表明，ChebyMA始终在参数-精度帕累托前沿上优于全参数微调、LoRA、TLoRA和StelLA。ChebyMA在所有测试数据集上均显著超越其他方法，验证了其纯向量化计算下具备通用性的坚实理论基础。
原文摘要 · Abstract (English)
The paper presents a new parameter-efficient adaptation method called ChebyMA (Chebyshev Manifold Adaptation). ChebyMA adopts weight matrices through a multi-surface superposition of Chebyshev polynomial bases evaluated on learnable coordinates and combined via trainable coefficient matrices, replacing standard linear projections with highly expressive continuous function approximation. Theoretically, we establish an Approximation Expressivity Theorem, proving from the perspective of function approximation theory that single-manifold ChebyMA guarantees convergence in Frobenius norm error of reconstruction. Besides, drawing on Kolmogorov $n$-width intuition, we demonstrate the expressive advantages of multi-manifold superposition ($S > 1$) in decoupling high-dimensional complex features. Experimental results on Computer Vision CIFAR datasets(CIFAR-10, CIFAR-100)\cite{CIFAR} and Natural Language Processing (AG News, SST-2) datasets demonstrate that ChebyMA consistently achieves a superior parameter-accuracy Pareto front compared to standard full-parameter fine-tuning, LoRA\cite{LoRA}, TLoRA\cite{TLoRA}, and StelLA\cite{StelLA}. ChebyMA significantly outperforms other tested methods in tested datasets, validating its solid theoretical foundation for generality with purely vectorized computations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。