无需预知分组信息,用网络扩散实现自动稀疏学习。
Learning under Latent Group Sparsity via Diffusion on Networks
- 基于网络热流动力学设计可自适应分组的正则化方法。
- 扩散时间仅需对数级即可保证统计性能,计算高效。
- 适用于无预处理分组、数据驱动建网的场景。
机器学习中解释变量的群体或聚类结构普遍存在。本文提出一种无需事先知晓分组身份的稀疏学习方法,其核心是利用底层网络的拉普拉斯几何与社区结构,通过基于热流的局部网络动态构建惩罚项。该惩罚项在Lasso与组Lasso之间插值,扩散时间作为调节参数,当网络分组结构弱时自动退化为Lasso。我们还提出数据驱动构建网络的流程,避免了耗时的变量聚类等预处理步骤。理论证明该方法具有良好的泛化性能,样本复杂度上界表明只需对数级扩散时间即可满足多数情形。方法与统计物理中的高斯自由场、随机块模型有深层联系,为利用数据的几何、动态与随机结构解决经典学习任务提供了新思路。
原文摘要 · Abstract (English)
Group or cluster structure on explanatory variables in machine learning problems is a very general phenomenon, which has attracted broad interest from practitioners and theoreticians alike. In this work we contribute an approach to sparse learning under such group structure, that does not require prior information on the group identities. Our paradigm is motivated by the Laplacian geometry of an underlying network with a related community structure, and proceeds by directly incorporating this into a penalty that is effectively computed via a heat-flow-based local network dynamics. The proposed penalty interpolates between the lasso and the group lasso penalties, the runtime of the heat-flow dynamics being the interpolating parameter. As such it can automatically default to lasso when the group structure reflected in the Laplacian is weak. In fact, we demonstrate a data-driven procedure to construct such a network based on the available data. Notably, we dispense with computationally intensive pre-processing involving clustering of variables, spectral or otherwise. Our technique is underpinned by rigorous theorems that guarantee its effective performance and provide bounds on its sample complexity. In particular, in a wide range of settings, it provably suffices to run the diffusion for time that is only logarithmic in the problem dimensions. We explore in detail the interfaces of our approach with key statistical physics models in network science, such as the Gaussian Free Field and the Stochastic Block Model. Our work raises the possibility of applying similar diffusion-based techniques to classical learning tasks, exploiting the interplay between geometric, dynamical and stochastic structures underlying the data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。