自适应平滑的朴素贝叶斯模型,提升高类别数据下的概率准确性与校准性。
Hierarchical Empirical-Bayes Naive Bayes: Minimax Smoothing and Calibration with AODE Extension

- 基于层次经验贝叶斯框架,动态学习每个类-特征的先验浓度,实现信息共享。
- 在31个数据集上平均排名最优,高类别数据下对数损失降低达22.1%。
- 适合需要精准概率输出和良好校准性的场景,如医疗、金融决策系统。
朴素贝叶斯(NB)分类器在类别型数据中仍为标准选择,但其常用平滑方法(如Laplace、Lidstone、Krichevsky-Trofimov、m-estimate)均采用固定平滑强度,忽略特征基数、样本量与类别不平衡,导致现代高基数表格数据中存在不可消除偏差。本文提出分层经验贝叶斯朴素贝叶斯(HEB-NB),每个类-特征条件概率通过可数据自适应学习的狄利克雷先验进行平滑,实现跨类信息共享的同时保持闭式推断。进一步引入HEB平均一依赖估计器(HEB-AODE),证明自适应平滑可自然扩展至NB的结构松弛。理论上,建立非渐近ℓ₁误差界,匹配经验分布极小极大率并带可消失的数据自适应偏差,且给出紧致的Laplace下界,实现有限样本风险严格分离。通过总变差张量化导出插件式超出贝叶斯风险界及群体顶1期望校准误差(ECE)推论。实验表明,在31个UCI与OpenML基准上,HEB-NB在概率指标上取得最佳平均弗里德曼秩,高基数数据上对数损失最高降低22.1%,HEB-AODE持续优于原始AODE。结合互信息加权,顶1 ECE下降41%-70%,显著提升概率准确性和校准性。
原文摘要 · Abstract (English)
The Naive Bayes (NB) classifier remains a standard choice for categorical data, yet its widely used smoothing rules, such as Laplace, Lidstone, Krichevsky-Trofimov, and the $m$-estimate, all prescribe a fixed smoothing strength that ignores feature cardinality, sample size, and class imbalance, inducing a non-vanishing bias on modern high-cardinality tabular data. We propose hierarchical empirical-Bayes Naive Bayes (HEB-NB), in which each class-feature conditional probability is smoothed by a Dirichlet prior whose concentration is learned data-adaptively via Type-II maximum likelihood, enabling principled information sharing across classes while retaining closed-form inference. We further introduce HEB average one-dependence estimators (HEB-AODE), showing that the adaptive smoothing transfers cleanly to structural relaxations of NB. Theoretically, we establish a non-asymptotic $\ell_1$ error bound for HEB-NB matching the empirical-distribution minimax rate plus a vanishing data-adaptive bias, together with a matching Laplace-tight lower bound that yields a finite-sample, risk-level strict separation from Laplace. We further derive a plug-in excess Bayes-risk bound via total-variation tensorization and a population top-1 expected calibration error (ECE) corollary. Empirically, across 31 UCI and OpenML benchmarks, HEB-NB attains the best average Friedman rank on probabilistic metrics, with up to 22.1% log-loss reductions on high-cardinality datasets and consistent improvements of HEB-AODE over vanilla AODE. Combining HEB-NB with mutual-information weighting reduces top-1 ECE by 41%-70%, demonstrating substantial gains in probabilistic accuracy and calibration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。