无需矩阵存储，通过向量乘法快速估算矩阵范数，提升模型训练与安全性能。
Matrix-Free Two-to-Infinity and One-to-Two Norms Estimation
- 基于哈奇森估计器改进，仅需矩阵-向量乘法
- 理论证明算法复杂度优于传统方法
- 适用于深度学习正则化与推荐系统抗攻击
本文提出新的随机化算法，在无需矩阵存储的条件下，仅通过矩阵-向量乘法估算两到无穷范数和一到二范数。方法基于哈奇森对角估计器及其改进版Hutch++的适配。我们为两种变体提供了最优者复杂度上界。实验表明，该算法在图像分类任务中可用于基于雅可比的正则化，有效提升深度神经网络训练性能；同时可应用于推荐系统，缓解对抗攻击影响。
原文摘要 · Abstract (English)
In this paper, we propose new randomized algorithms for estimating the two-to-infinity and one-to-two norms in a matrix-free setting, using only matrix-vector multiplications. Our methods are based on appropriate modifications of Hutchinson's diagonal estimator and its Hutch++ version. We provide oracle complexity bounds for both modifications. We further illustrate the practical utility of our algorithms for Jacobian-based regularization in deep neural network training on image classification tasks. We also demonstrate that our methodology can be applied to mitigate the effect of adversarial attacks in the domain of recommender systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。