PRISM通过谱形变实现自适应优化，用少量计算提升收敛速度。
PRISM: Structured Optimization via Anisotropic Spectral Shaping
- 用创新增强的极分解构建低秩预条件器，融合部分二阶信息。
- 在高方差方向抑制更新，在信号主导方向保留强度，提升优化效率。
- 无需额外内存，计算开销极小，适合部署于资源受限场景。
我们提出 PRISM，一种增强一阶谱下降方法（如 Muon）的优化器，通过创新增强的极分解构建高效低秩准二阶预条件器。该机制实现各向异性谱形变，自适应抑制高方差子空间中的更新，同时保留信号主导方向的更新强度。关键优势在于仅需极少计算开销，且相比一阶基线零额外内存开销。PRISM为谱优化范式引入曲率自适应特性提供了实用策略。
原文摘要 · Abstract (English)
We propose PRISM, an optimizer that enhances first-order spectral descent methods like Muon with partial second-order information. It constructs an efficient, low-rank quasi-second-order preconditioner via innovation-augmented polar decomposition. This mechanism enables PRISM to perform anisotropic spectral shaping, which adaptively suppresses updates in high-variance subspaces while preserving update strength in signal-dominated directions. Crucially, this is achieved with minimal computational overhead and zero additional memory compared to first-order baselines. PRISM demonstrates a practical strategy for integrating curvature-adaptive properties into the spectral optimization paradigm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。