同时压缩数据量和维度，保持分布不变且效率更高
Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality
- 双阶段压缩：先降维再优化低维编码集
- 压缩率更高，下游任务性能相当或更好
- 适合大规模高维数据的高效处理
现有分布压缩方法仅减少数据样本数量，但现代数据集往往兼具海量样本与高维特征。本文提出双边分布压缩（BDC），一种在样本数和维度两个方向同时压缩的两阶段框架，整体时间与内存复杂度对数据规模呈线性关系。BDC核心是解码最大均值差异（DMMD），用于衡量原始数据与从低维潜在空间重构的压缩集之间的分布差异。BDC分两步：(i) 使用重建最大均值差异（RMMD）学习低维投影；(ii) 通过编码最大均值差异（EMMD）优化潜在压缩集。该过程最小化DMMD，确保压缩集忠实反映原始分布。实验表明，BDC在远低于环境空间压缩的成本下，实现相当或更优的下游任务性能，且压缩率显著提升。
原文摘要 · Abstract (English)
Existing distribution compression methods reduce the number of observations in a dataset by minimising the Maximum Mean Discrepancy (MMD) between original and compressed sets, but modern datasets are often large in both sample size and dimensionality. We propose Bilateral Distribution Compression (BDC), a two-stage framework that compresses along both axes while preserving the underlying distribution, with overall linear time and memory complexity in dataset size and dimension. Central to BDC is the Decoded MMD (DMMD), which we introduce to quantify the discrepancy between the original data and a compressed set decoded from a low-dimensional latent space. BDC proceeds by (i) learning a low-dimensional projection using the Reconstruction MMD (RMMD), and (ii) optimising a latent compressed set with the Encoded MMD (EMMD). We show that this procedure minimises the DMMD, guaranteeing that the compressed set faithfully represents the original distribution. Experiments show that BDC can achieve comparable or superior downstream task performance to ambient-space compression at substantially lower cost and with significantly higher rates of compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。