用负相关提升机器学习性能,突破传统独立假设限制
Negative Dependence as a toolbox for machine learning : review and new developments
- 将负相关作为统一方法论,整合多种随机模型用于学习
- 在优化、采样、降维等任务中超越独立性方法表现
- 适合对概率建模与算法设计感兴趣的研究者
负相关正成为突破传统独立性假设、提升机器学习能力的关键驱动力。近二十年来,其在优化、采样、降维和稀疏信号恢复等基础问题中展现出优越性能,显著优于依赖统计独立性的现有方法。最具代表性的是确定性点过程(DPPs),源于量子理论,但扰动格点模型、强雷利测度、随机函数零点等也逐渐受到关注。本文系统回顾该领域发展,重点介绍作者近年在蒙特卡洛方法、核心子集(coresets)、随机梯度下降、随机网络、信号处理及量子计算关联方面的应用成果。不同于以往聚焦单一模型的综述,本文将负相关视为整体方法论,涵盖多种模型及其广泛应用场景,提供了一个全面且独特的视角。
原文摘要 · Abstract (English)
Negative dependence is becoming a key driver in advancing learning capabilities beyond the limits of traditional independence. Recent developments have evidenced support towards negatively dependent systems as a learning paradigm in a broad range of fundamental machine learning challenges including optimization, sampling, dimensionality reduction and sparse signal recovery, often surpassing the performance of current methods based on statistical independence. The most popular negatively dependent model has been that of determinantal point processes (DPPs), which have their origins in quantum theory. However, other models, such as perturbed lattice models, strongly Rayleigh measures, zeros of random functions have gained salience in various learning applications. In this article, we review this burgeoning field of research, as it has developed over the past two decades or so. We also present new results on applications of DPPs to the parsimonious representation of neural networks. In the limited scope of the article, we mostly focus on aspects of this area to which the authors contributed over the recent years, including applications to Monte Carlo methods, coresets and stochastic gradient descent, stochastic networks, signal processing and connections to quantum computation. However, starting from basics of negative dependence for the uninitiated reader, extensive references are provided to a broad swath of related developments which could not be covered within our limited scope. While existing works and reviews generally focus on specific negatively dependent models (e.g. DPPs), a notable feature of this article is that it addresses negative dependence as a machine learning methodology as a whole. In this vein, it covers within its span an array of negatively dependent models and their applications well beyond DPPs, thereby putting forward a very general and rather unique perspective.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。