arXiv:2604.04726stat.MLcs.LG2026-04

用新型加速算法提升张量广义线性模型的计算效率与精度

A Muon-Accelerated Algorithm for Low Separation Rank Tensor Generalized Linear Models

论文配图:A Muon-Accelerated Algorithm for Low Separation Rank Tensor Generalized Linear Models
图 1 · 摘自论文原文
  • 在原有分块坐标下降框架中,用牛顿-舒尔优化动量替代重复正交投影
  • 合成数据上迭代次数和实际耗时均减少,估计与预测误差更低
  • 适合处理高维医学影像等多维张量数据的高效建模任务

张量数据在多维信号与成像问题中普遍存在,如生物医学成像。当引入广义线性模型(GLMs)时,直接向量化会破坏其多向结构,导致高维且病态的估计问题。为应对挑战,低分离秩(LSR)分解通过在系数张量上施加低秩多线性结构来降低模型复杂度。代表性方法是低分离秩张量回归(LSRTR),采用分块坐标下降,并通过反复的基于QR的投影强制因子矩阵正交。然而,重复投影步骤计算开销大,收敛缓慢。为实现更高效的估计与分类,本文提出LSRTR-M,将穆子(Muon,即牛顿-舒尔正交化动量)更新引入LSRTR框架。具体而言,保持原有分块坐标方案,但以穆子步骤替代基于投影的因子更新。在合成的线性、逻辑和泊松型LSR-TGLMs上,LSRTR-M在迭代次数和实际运行时间上均更快收敛,同时达到更低的归一化估计误差与预测误差。在Vessel MNIST 3D任务中,进一步提升了计算效率,且分类性能保持竞争力。

原文摘要 · Abstract (English)

Tensor-valued data arise naturally in multidimensional signal and imaging problems, such as biomedical imaging. When incorporated into generalized linear models (GLMs), naive vectorization can destroy their multi-way structure and lead to high-dimensional, ill-posed estimation. To address this challenge, Low Separation Rank (LSR) decompositions reduce model complexity by imposing low-rank multilinear structure on the coefficient tensor. A representative approach for estimating LSR-based tensor GLMs (LSR-TGLMs) is the Low Separation Rank Tensor Regression (LSRTR) algorithm, which adopts block coordinate descent and enforces orthogonality of the factor matrices through repeated QR-based projections. However, the repeated projection steps can be computationally demanding and slow convergence. Motivated by the need for scalable estimation and classification from such data, we propose LSRTR-M, which incorporates Muon (MomentUm Orthogonalized by Newton-Schulz) updates into the LSRTR framework. Specifically, LSRTR-M preserves the original block coordinate scheme while replacing the projection-based factor updates with Muon steps. Across synthetic linear, logistic, and Poisson LSR-TGLMs, LSRTR-M converges faster in both iteration count and wall-clock time, while achieving lower normalized estimation and prediction errors. On the Vessel MNIST 3D task, it further improves computational efficiency while maintaining competitive classification performance.

张量模型加速算法医学影像广义线性模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。