让粒子数据压缩后仍保持聚类结构,提升压缩比4倍
Preserving Clusters in Error-Bounded Lossy Compression of Scientific Particle Data
- 通过修正重建数据中的关键连接关系,保证聚类成员不变
- 在多个科学数据集上实现最高4倍压缩率提升
- 适用于宇宙学、分子动力学等领域的大型模拟数据
宇宙学、分子动力学和流体动力学的粒子模拟产生大规模数据集,其存储、传输和分析越来越依赖有损压缩。然而现有压缩器通常仅限制点对点位置误差,无法保证由粒子坐标推导出的结构(如单链接聚类,即朋友邻域算法)的保真度。即使在距离阈值附近的微小坐标扰动,也可能破坏真实连接或产生虚假连接,导致聚类分裂或合并。本文提出一种与压缩器无关的校正技术,在保留原始点对点误差边界的前提下,确保单链接聚类成员关系不变。该方法作用于标准压缩器(如SZ3、ZFP、Draco、LCP)的重构输出,以紧凑的校正编辑流形式存储修正信息。核心观察是:聚类成员查询依赖连通分量而非全部邻近连接。基于此,设计了三种约束选择模式——易损对、安全组件和晕环森林,逐步减少校正时施加的约束。使用投影梯度下降修正重构坐标,消除选定的违规连接,同时满足原始误差上限。在宇宙学、分子动力学和流体动力学数据集上的实验表明,该方法在单GPU和分布式内存环境下均能保持聚类成员不变,并将压缩比提升最高达4倍,同时端到端吞吐量与配置紧致误差边界的基准压缩器相当。
原文摘要 · Abstract (English)
Scientific particle simulations in cosmology, molecular dynamics, and fluid dynamics produce large-scale datasets whose storage, movement, and analysis increasingly rely on lossy compression. However, existing compressors typically bound only pointwise position errors, providing no guarantee on the fidelity of structures derived from particle coordinates, such as single-linkage clustering (also known as Friends-of-Friends algorithm), where clusters are connected components of a proximity graph formed by linking particle pairs within a distance threshold. Even small coordinate perturbations near this threshold can break true links or create false links, thereby splitting or merging entire clusters. We propose a compressor-independent correction technique for preserving single-linkage cluster membership under lossy compression. Our method operates on reconstructed outputs from off-the-shelf compressors such as SZ3, ZFP, Draco, and LCP, and stores a compact corrective edit stream. Our key observation is that cluster-membership queries depend on connected components rather than the complete set of proximity links. Based on this observation, we introduce three constraint-selection modes, vulnerable-pair, safe-component, and halo-forest, that progressively reduce the constraints enforced during correction. Projected gradient descent then corrects the reconstructed coordinates to eliminate the selected violations while respecting the original pointwise error bound. Experiments on cosmology, molecular dynamics, and fluid dynamics datasets with single-GPU and distributed-memory implementations show that our method preserves cluster membership while improving compression ratio by up to 4$\times$ and maintains competitive end-to-end throughput compared to the same base compressors configured with sufficiently tight error bounds to preserve clustering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。