用神经网络与变分范数优化张量分解,实现更稀疏、平滑的多维数据恢复。
Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization
- 基于CP分解构建神经网络参数化张量模型,支持网格内外建模。
- 引入变分Schatten-p准范数,有效剪除冗余秩-1分量。
- 采用雅可比谱范数正则化,无需奇异值分解,适合隐式神经表示。
高阶张量能有效表示图像、视频等多维数据,通常具有低秩结构。低秩张量分解在机器学习和计算机视觉中至关重要,但传统方法如Tucker分解虽灵活却难解释;而CANDECOMP/PARAFAC(CP)分解结构自然且可解释,却难以获得稀疏解。本文利用CP分解的丰富特性,提出一种基于神经网络(NN)参数化的CP型低秩张量函数,用于隐式神经表示。该方法可实现网格内与网格外建模,并充分挖掘神经网络的非线性能力,同时具备过剩风险界的理论保证。为获得更稀疏的CP分解,引入变分Schatten-p准范数以剪除冗余的秩-1成分,并证明其为任意展开矩阵的Schatten-p准范数的通用上界。为提升平滑性,提出基于雅可比谱范数和Hutchinson迹估计器的正则化项,该方法无需奇异值分解(SVD),避免显式链式法则推导,可作为图像去噪中总变差(TV)正则化的替代方案,且天然适用于隐式神经表示。在图像修补、去噪及点云上采样等多维数据恢复任务中的大量实验表明,该方法优于现有最先进方法,展现出卓越的性能与通用性。代码已公开于 https://github.com/CZY-Code/CP-Pruner。
原文摘要 · Abstract (English)
Higher-order tensors are well-suited for representing multi-dimensional data, such as images and videos, which typically characterize low-rank structures. Low-rank tensor decomposition has become essential in machine learning and computer vision, but existing methods like Tucker decomposition offer flexibility at the expense of interpretability. The CANDECOMP/PARAFAC (CP) decomposition provides a natural and interpretable structure, while obtaining a sparse solutions remains challenging. Leveraging the rich properties of CP decomposition, we propose a CP-based low-rank tensor function parameterized by neural networks (NN) for implicit neural representation. This approach can model the tensor both on-grid and beyond grid, fully utilizing the non-linearity of NN with theoretical guarantees on excess risk bounds. To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components and prove that it serves as a common upper bound for the Schatten-p quasi-norms of arbitrary unfolding matrices. For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator. The proposed smoothness regularization is SVD-free and avoids explicit chain rule derivations. It can serve as an alternative to Total Variation (TV) regularization in image denoising tasks and is naturally applicable to implicit neural representation. Extensive experiments on multi-dimensional data recovery tasks, including image inpainting, denoising, and point cloud upsampling, demonstrate the superiority and versatility of our method compared to state-of-the-art approaches. The code is available at https://github.com/CZY-Code/CP-Pruner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。