提出新概率不等式，显著提升学习理论中泛化误差的分析精度。
Tighter Information-Theoretic Generalization Bounds via a Novel Class of Change of Measure Inequalities
- 基于数据处理不等式构建统一框架，生成更紧致的概率界
- 涵盖多种信息度量，实现比已有方法更强的泛化保证
- 适用于泛化分析、差分隐私等领域，简化复杂推导
变测度不等式将概率测度间的差异转化为事件概率的显式上界，在学习理论、信息论和统计学中具有重要作用。本文提出一种基于数据处理不等式的统一框架，导出一系列新颖且更紧致的变测度不等式。该框架覆盖广义信息度量，包括f-散度（含KL散度和χ²散度）、Rényi散度及α-互信息（含最大泄漏作为特例）。我们将这些结果应用于泛化误差分析、PAC-Bayesian理论、差分隐私与数据记忆性研究，获得了更强的保证，并以更简洁的推导恢复了现有最优结果。
原文摘要 · Abstract (English)
Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and statistics. We propose novel change of measure inequalities via a unified framework based on the data processing inequality, which is surprisingly elementary yet powerful enough to yield novel, tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We apply these results to generalization error analysis, PAC-Bayesian theory, differential privacy, and data memorization, obtaining stronger guarantees while recovering best-known results through simplified analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。