用稀疏核心集加速概率分布优化，计算量从N²降至N¹·⁵
Thinned Mean Field Langevin Dynamics
- 每个粒子只与√N大小的核心集互动，降低计算开销
- 在神经网络训练等任务中保持与原方法相当的收敛效果
- 适合需要高效贝叶斯推断和分布优化的场景
许多重要学习任务可表述为在概率分布空间上最小化熵正则化目标。均场朗之万动力学（MFLD）通过将最优解视为麦凯恩-弗拉索夫过程的平稳分布，实现该类问题的数值求解，使用N个粒子进行模拟。然而，其交互计算复杂度为O(N²)。受核稀疏化（kernel thinning）启发，本文提出KT-MFLD，使每个粒子仅与大小为O(N¹·⁵)的稀疏核心集交互，将复杂度降至O(N¹·⁵)，在温和正则性条件下仍保持与MFLD相同的收敛保证（对数因子内）。理论分析在学生-教师神经网络训练、最大均值差异量化及后贝叶斯框架下的预测性后验计算等任务中得到实证支持。
原文摘要 · Abstract (English)
Several important learning tasks can be formulated as minimizing an entropy-regularized objective over an appropriate space of probability distributions. Mean-field Langevin dynamics (MFLD) facilitate computation in this general context, casting the minimizer as the invariant distribution of a McKean--Vlasov process, which can be numerically discretized using $N$ particles and thus simulated. However, simulating this interacting particle system has computational complexity of order $N^2$. Motivated by recent research into \emph{kernel thinning}, we propose \texttt{KT-MFLD}, in which each particle interacts only with a thinned particle coreset of size $\mathcal{O}(N^{\frac{1}{2}})$. \texttt{KT-MFLD} thus reduces the computational complexity to order $N^{\frac{3}{2}}$ while, under mild regularity conditions, achieving the same convergence guarantees (up to logarithmic factors) as MFLD. Our theoretical analysis is empirically confirmed on tasks including the training of student-teacher neural networks, quantization with maximum mean discrepancy, and computation of predictively-oriented posteriors in a post-Bayesian framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。