提出新模型提升高维不完整数据的低秩表示效果
Sharpness-aware Second-order Latent Factor Model for High-dimensional and Incomplete Data
- 用海森向量积获取二阶信息，增强优化稳定性
- 引入尖锐度项改善损失曲面平坦度，提升泛化能力
- 在多个工业数据集上优于现有最佳方法
第二阶潜在因子（SLF）模型是一类有效的低秩表示学习方法，可从高维不完整（HDI）数据中提取节点间交互模式。然而，其优化因双线性与非凸特性而极具挑战。最近提出的尖锐度感知最小化（SAM）通过寻找非凸目标的平坦局部极小值，提升了表示学习模型的泛化性能。为应对这一挑战，我们提出尖锐度感知的SLF（SSLF）模型。SSLF融合两个关键思想：(1) 通过海森向量积获取二阶信息；(2) 利用设计的海森向量积将尖锐度项注入曲率（海森矩阵）。在多个工业数据集上的实验表明，所提模型始终优于当前最优基线。
原文摘要 · Abstract (English)
Second-order Latent Factor (SLF) model, a class of low-rank representation learning methods, has proven effective at extracting node-to-node interaction patterns from High-dimensional and Incomplete (HDI) data. However, its optimization is notoriously difficult due to its bilinear and non-convex nature. Sharpness-aware Minimization (SAM) has recently proposed to find flat local minima when minimizing non-convex objectives, thereby improving the generalization of representation-learning models. To address this challenge, we propose a Sharpness-aware SLF (SSLF) model. SSLF embodies two key ideas: (1) acquiring second-order information via Hessian-vector products; and (2) injecting a sharpness term into the curvature (Hessian) through the designed Hessian-vector products. Experiments on multiple industrial datasets demonstrate that the proposed model consistently outperforms state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。