提出最优分布估计方法，解决小样本下高概率误差难题。
Estimation of discrete distributions in relative entropy, and the deviations of the missing mass
- 用数据自适应平滑改进经典拉普拉斯估计
- 首次给出缺失质量的紧致高概率上界
- 适合小样本、稀疏分布场景的高效估计
研究从独立同分布样本估计有限字母表上的分布，以相对熵（Kullback-Leibler散度）衡量精度。虽然期望风险的最优界已知，但高概率保证仍不清晰。首先分析经典的拉普拉斯（加一）估计器，获得其性能的上下界匹配，证明其在无置信度依赖估计器中是最优的。随后刻画了极小极大高概率风险，并发现由简单置信度依赖平滑技术实现。值得注意的是，最优非渐近风险相比理想渐近率多出一个对数因子。接着，在字母表大小超过样本量的场景下，研究适应分布稀疏性的方法。引入一种基于数据的自适应平滑估计器，其高概率风险依赖于两个有效稀疏性参数。分析中还导出了缺失质量的紧致高概率上界。
原文摘要 · Abstract (English)
We study the problem of estimating a distribution over a finite alphabet from an i.i.d. sample, with accuracy measured in relative entropy (Kullback-Leibler divergence). While optimal bounds on the expected risk are known, high-probability guarantees remain less well-understood. First, we analyze the classical Laplace (add-one) estimator, obtaining matching upper and lower bounds on its performance and establishing its optimality among confidence-independent estimators. We then characterize the minimax-optimal high-probability risk and show that it is achieved by a simple confidence-dependent smoothing technique. Notably, the optimal non-asymptotic risk incurs an additional logarithmic factor compared to the ideal asymptotic rate. Next, motivated by regimes in which the alphabet size exceeds the sample size, we investigate methods that adapt to the sparsity of the underlying distribution. We introduce an estimator using data-dependent smoothing, for which we establish a high-probability risk bound depending on two effective sparsity parameters. As part of our analysis, we also derive a sharp high-probability upper bound on the missing mass.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。