针对核方法矩阵病态问题，提出自适应谱衰减正则化，提升分类准确率。
Differential Spectral Damping Gap Adaptive Regularization for Ill-Conditioned Kernel Methods
- 根据谱间隙大小动态调整正则化强度，区分可靠与不可靠特征方向。
- 在GINA数据集上提升分类准确率4.8个百分点，最高达10.4个百分点。
- 适用于高维病态矩阵场景，为实际应用提供明确使用边界。
需要矩阵求逆的核方法，尤其是最小二乘双支持向量机（LSTSVM），其系统矩阵存在指数级衰减的特征值，导致严重病态问题。标准Tikhonov正则化对所有特征向量施加均匀抑制，无法区分可靠性。本文提出差分谱衰减（DSD）正则化，依据局部谱间隙结构自适应调节惩罚力度：保留具有大谱间隙的特征向量（根据Davis-Kahan扰动理论可信），同时强烈抑制小谱间隙的特征向量（方向已受污染无法恢复）。基于Davis-Kahan sin(Θ)定理，系统推导出可靠性感知衰减函数的设计要求，并选择指数形式以保证光滑性、可微性及自然饱和特性。与经过公平优化的基线方法（包括接受同等优化的梯度优化Tikhonov）进行严格配对测试，结果表明，仅使用原理性谱初始化，DSD在真实世界GINA数据集（d=970，Cohen's d=4.49，p<0.0001）上使分类准确率提升4.8个百分点，在d=200时提升10.4个百分点，在Madelon数据集（d=500）上提升2.6个百分点。在流形数据的预图像重构中，当扰动噪声水平为p=0.99时，DSD性能与Tikhonov持平，低噪声下略逊，但两者均将原始反演误差降低66倍。本文精确刻画了适用范围（d≥100，条件数>10³），并指出简单方法足以应对的情况，为实践者提供清晰部署指引。
原文摘要 · Abstract (English)
Kernel methods requiring matrix inversion -- particularly Least-Squares Twin Support Vector Machines (LSTSVM) -- suffer from exponential eigenvalue decay in their system matrices, producing severely ill-conditioned problems where standard Tikhonov regularization applies uniform damping regardless of eigenvector reliability. We propose Differential Spectral Damping (DSD), a regularization formula that adapts its penalty to localized eigengap structure: preserving eigenvectors with large spectral gaps (reliable per Davis-Kahan perturbation theory) while aggressively suppressing those with small gaps (directionally corrupted beyond recovery). We motivate DSD through a principled design procedure grounded in the Davis-Kahan $\sin(Θ)$ theorem, systematically deriving the requirements for a reliability-aware damping function and selecting the exponential form for its smoothness, differentiability, and natural saturation properties. Through rigorous paired testing with fairly optimized baselines (including gradient-optimized Tikhonov receiving equal optimization opportunity), we demonstrate that DSD improves LSTSVM classification accuracy by +4.8 percentage points on real-world GINA ($d=970$, Cohen's $d = 4.49$, $p < 0.0001$), +10.4 percentage points at $d=200$, and +2.6 percentage points on Madelon ($d=500$) -- all using only principled spectral initialization while Tikhonov receives grid search. For pre-image reconstruction on manifold data, DSD ties Tikhonov at high perturbation noise ($p=0.99$) but slightly underperforms at lower noise levels; both reduce naive inversion error by $66\times$. We characterize the precise operating regime ($d \geq 100$, condition number $> 10^3$) and document where simpler methods suffice, providing practitioners with clear deployment guidance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。