提出可高效计算的平滑度正则化方法，提升概率电路泛化能力。
Tractable Sharpness-Aware Learning of Probabilistic Circuits
- 基于海森矩阵迹设计平滑度正则项，解决概率电路过拟合问题
- 在合成与真实数据集上均实现更平坦的极小值点和更好泛化性能
- 适用于数据量少时的生成模型训练，尤其适合需要精确推断的场景
概率电路（PCs）是一类支持多种查询下精确且高效推断的生成模型。尽管近期进展使深度且表达能力强的PCs成为可能，但其高容量常导致过拟合，尤其在数据有限时。本文从对数似然景观角度分析了过拟合现象，发现其根源常是收敛至泛化能力差的尖锐极小值。受神经网络中尖锐感知最小化启发，我们提出一种基于海森矩阵的正则化方法。关键贡献在于，我们证明了对数似然的海森矩阵迹——通常在深度神经网络中难以计算的平滑度代理——可在概率电路中高效求解。最小化该迹值可导出基于梯度范数的正则化项，实现EM算法的闭式参数更新，并与基于梯度的学习方法无缝集成。在合成与真实世界数据集上的实验表明，该方法能稳定引导PCs趋向更平坦的极小值，显著提升泛化性能。
原文摘要 · Abstract (English)
Probabilistic Circuits (PCs) are a class of generative models that allow exact and tractable inference for a wide range of queries. While recent developments have enabled the learning of deep and expressive PCs, this increased capacity can often lead to overfitting, especially when data is limited. We analyze PC overfitting from a log-likelihood-landscape perspective and show that it is often caused by convergence to sharp optima that generalize poorly. Inspired by sharpness aware minimization in neural networks, we propose a Hessian-based regularizer for training PCs. As a key contribution, we show that the trace of the Hessian of the log-likelihood-a sharpness proxy that is typically intractable in deep neural networks-can be computed efficiently for PCs. Minimizing this Hessian trace induces a gradient-norm-based regularizer that yields simple closed-form parameter updates for EM, and integrates seamlessly with gradient based learning methods. Experiments on synthetic and real-world datasets demonstrate that our method consistently guides PCs toward flatter minima, improves generalization performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。