通过新正则化方法实现稀疏字典学习,显著减少冗余原子。
A Unified Probabilistic Framework for Dictionary Learning with Parsimonious Activation
- 引入行范数惩罚,让整行系数归零以减少激活原子
- 重建误差降低20%,仅用不到十分之一的字典原子
- 理论完备且适合需要高效稀疏表示的研究者
传统字典学习通常以L1正则化信号重构问题形式建模。尽管近期研究引入了判别性、层次化或生成式结构,但多数方法仍依赖于对单个样本的稀疏性激励,忽略了原子在样本间的共享机制,导致字典冗余且性能欠佳。本文提出基于系数矩阵行向L∞范数的稀疏促进正则化项,该惩罚使系数矩阵整行趋于零,从而减少整个数据集上被激活的字典原子数量。该正则化由具有贝塔-伯努利先验的概率模型推导而来,提供贝叶斯解释,并将正则化参数与先验分布关联。进一步建立了最优超参数选择的理论计算方法,并将本方法与最小描述长度、贝叶斯模型选择及路径子学习相联系。在基准数据集上的大量实验表明,该方法在重建质量上取得显著提升(均方根误差降低20%),同时实现更强的表示稀疏性,仅使用不到十分之一的可用字典原子,且实证验证了理论分析的正确性。
原文摘要 · Abstract (English)
Dictionary learning is traditionally formulated as an $L_1$-regularized signal reconstruction problem. While recent developments have incorporated discriminative, hierarchical, or generative structures, most approaches rely on encouraging representation sparsity over individual samples that overlook how atoms are shared across samples, resulting in redundant and sub-optimal dictionaries. We introduce a parsimony promoting regularizer based on the row-wise $L_\infty$ norm of the coefficient matrix. This additional penalty encourages entire rows of the coefficient matrix to vanish, thereby reducing the number of dictionary atoms activated across the dataset. We derive the formulation from a probabilistic model with Beta-Bernoulli priors, which provides a Bayesian interpretation linking the regularization parameters to prior distributions. We further establish theoretical calculation for optimal hyperparameter selection and connect our formulation to both Minimum Description Length, Bayesian model selection and pathlet learning. Extensive experiments on benchmark datasets demonstrate that our method achieves substantially improved reconstruction quality (with a 20\% reduction in RMSE) and enhanced representation sparsity, utilizing fewer than one-tenth of the available dictionary atoms, while empirically validating our theoretical analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。