提出新方法构建预测性压缩数据集,适用于非参数模型。
Predictive Coresets
- 用随机后验匹配全量与压缩数据的预测分布来生成权重
- 在随机划分和密度估计任务中保持与全量数据相近的预测性能
- 适合处理大规模非参数模型的压缩数据问题
现代数据分析常面临包含数十万观测的大规模数据集,传统推断算法计算成本过高。压缩数据集(Coresets)是一种选择小样本子集的方法,能在保持学习性能的同时降低计算量。传统方法通过最小化全量数据与加权数据似然函数之间的KL散度确定权重,但在非参数模型中因似然函数难以计算而失效。本文提出一种基于变分思想的新方法,利用随机后验,寻找权重使全量数据与缩减数据的未知后验预测分布相匹配。该方法基于预测递归,适用于非参数先验,具有通用性。我们在随机划分和密度估计等多种问题上评估了所提压缩数据构造方法,结果表明其能有效保持与全量数据相当的预测性能。
原文摘要 · Abstract (English)
Modern data analysis often involves massive datasets with hundreds of thousands of observations, making traditional inference algorithms computationally prohibitive. Coresets are selection methods designed to choose a smaller subset of observations while maintaining similar learning performance. Conventional coreset approaches determine these weights by minimizing the Kullback-Leibler (KL) divergence between the likelihood functions of the full and weighted datasets; as a result, this makes them ill-posed for nonparametric models, where the likelihood is often intractable. We propose an alternative variational method which employs randomized posteriors and finds weights to match the unknown posterior predictive distributions conditioned on the full and reduced datasets. Our approach provides a general algorithm based on predictive recursions suitable for nonparametric priors. We evaluate the performance of the proposed coreset construction on diverse problems, including random partitions and density estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。