arXiv:2502.20293cs.LG2025-02中稿 · publication in AI …

用图注意力机制高效筛选关键数据,大幅压缩数据集仍保持模型性能。

Scalable Graph Attention-based Instance Selection via Mini-Batch Sampling and Hierarchical Hashing

  • 基于图注意力机制,通过结构关系识别重要样本。
  • 在39个数据集上实现超96%压缩率,性能不降反升。
  • 适合大规模数据清洗与高维复杂数据的高效处理场景。

实例选择(IS)旨在减少数据集规模的同时保留关键信息,随着数据量达百万级愈发重要。现有方法难以捕捉高维空间中的复杂关系且扩展性差。本文提出一种基于图注意力的实例选择方法(GAIS),利用注意力机制通过图表示中的结构关系识别重要实例。提出两种可扩展的图构建方案:基于距离的最小批量采样技术,通过分批处理实现与数据集大小无关的复杂度;以及层次哈希方法,通过随机投影实现高效相似性计算。最小批量方法通过分层采样保持类别分布,层次哈希方法通过单层、多层和多视角变体在不同粒度上捕捉关系。在39个数据集上的实验表明,GAIS实现超过96%的压缩率,同时性能优于或等同于现有最佳方法。结果表明,基于距离的最小批量方法在大规模数据上效率最优,多视角变体在复杂高维数据上表现更佳,验证了注意力重要性评分能有效识别维持决策边界的实例,避免耗时的成对比较。

原文摘要 · Abstract (English)

Instance selection (IS) addresses the critical challenge of reducing dataset size while keeping informative characteristics, becoming increasingly important as datasets grow to millions of instances. Current IS methods often struggle with capturing complex relationships in high-dimensional spaces and scale with large datasets. This paper introduces a graph attention-based instance selection (GAIS) method that uses attention mechanisms to identify informative instances through their structural relationships in graph representations. We present two approaches for scalable graph construction: a distance-based mini-batch sampling technique that achieves dataset-size-independent complexity through strategic batch processing, and a hierarchical hashing approach that enables efficient similarity computation through random projections. The mini-batch approach keeps class distributions through stratified sampling, while the hierarchical hashing method captures relationships at multiple granularities through single-level, multi-level, and multi-view variants. Experiments across 39 datasets show that GAIS achieves reduction rates above 96\% while maintaining or improving model performance relative to state-of-the-art IS methods. The findings show that the distance-based mini-batch approach offers an optimal efficiency for large-scale datasets, while multi-view variants excel on complex, high-dimensional data, demonstrating that attention-based importance scoring can effectively identify instances important for maintaining decision boundaries while avoiding computationally prohibitive pairwise comparisons.

实例选择图神经网络数据压缩高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。