arXiv:2606.21080stat.MLcs.LG2026-06

解决冗余变量下贝叶斯模型平均的后验不稳定问题

Bayesian Model Averaging under Predictor Redundancy via Density-Ratio Posterior Compression

  • 用密度比压缩后验支持集,生成更简洁的报告区域
  • 通过总变差和KL散度量化信息损失,确保预测稳定
  • 适合处理高维回归中变量冗余、后验分散的问题

在支持索引回归中,贝叶斯模型平均会产生活跃预测变量支持集的后验分布。当存在预测变量冗余时,后验质量会分散在多个几乎可互换的支持集上,导致精确支持总结不稳或难以解释,即使预测表现稳定。本文研究如何在不改变贝叶斯目标的前提下,报告已拟合的贝叶斯模型平均后验。报告使用支持空间中的硬区域或软区域,其压缩报告律通过显式密度比与参考后验比较。该密度比可计算总变差和Kullback-Leibler偏差,为有界预测摘要提供误差界,提供保留质量诊断和后备权重诊断。框架涵盖固定硬区域、度量球区域、后验聚类区域及合并剪枝区域词典。我们推导了这些区域报告的精确误差公式和验证界,并给出少数区域可替代长列表个体支持的条件。模拟结果表明,区域报告通常更短更清晰,同时保留主要后验信息,密度比诊断能揭示信息丢失程度。

原文摘要 · Abstract (English)

Bayesian model averaging in support-indexed regression induces a posterior distribution over active predictor supports. Under predictor redundancy, posterior mass can spread across many nearly interchangeable supports, making exact-support summaries unstable or hard to interpret even when prediction is stable. We study how to report an already fitted Bayesian model averaging posterior without changing the Bayesian target. A report uses hard or soft regions of support space, and its compressed reporting law is compared with the reference posterior through an explicit density ratio. This ratio gives computable total-variation and Kullback--Leibler distortion, bounds for bounded predictive summaries, retained-mass diagnostics, and fallback-weight diagnostics. The framework covers fixed hard regions, metric-ball regions, posterior-cluster regions, and pooled-pruned region dictionaries. We prove exact error formulas and validation bounds for these region reports, and give conditions under which a few regions can replace a long list of individual supports. In simulations, our region reports often give shorter and clearer summaries while preserving the main posterior information, and the density-ratio diagnostics show when too much information has been lost.

贝叶斯统计模型平均变量选择后验压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。