arXiv:2502.15032cs.LG2025-02AAAI被引 2

用轻量Transformer高效建模地理空间表格数据,兼顾精度与速度。

GeoAggregator: An Efficient Transformer Model for Geo-Spatial Tabular Data

  • 基于高斯偏置局部注意力和全局位置感知,捕捉地理相关性与异质性。
  • 在多个数据集上表现优于或接近顶尖模型,且模型体积更小。
  • 适合需要高效处理大规模地理数据的研究者与工程师使用。

用深度学习建模地理空间表格数据正逐渐成为传统统计与机器学习方法的有力替代。然而,随着数据规模增长,现有深度学习模型常面临可扩展性和灵活性不足的问题。为此,本文提出GeoAggregator,一种基于Transformer架构、专为地理空间表格数据设计的高效轻量级算法。该模型通过高斯偏置局部注意力和全局位置感知,显式建模空间自相关性和空间异质性。此外,引入一种基于笛卡尔积的新注意力机制,在控制模型规模的同时保持强大表达能力。我们在合成与真实地理空间数据集上,将GeoAggregator与空间统计模型、XGBoost及多个先进地理空间深度学习方法进行对比。结果表明,其在几乎所有数据集上均达到最佳或第二佳性能。模型体积显著减小,兼具高效性与可扩展性。消融实验验证了高斯偏置与笛卡尔注意力机制的有效性,为后续优化提供依据。

原文摘要 · Abstract (English)

Modeling geospatial tabular data with deep learning has become a promising alternative to traditional statistical and machine learning approaches. However, existing deep learning models often face challenges related to scalability and flexibility as datasets grow. To this end, this paper introduces GeoAggregator, an efficient and lightweight algorithm based on transformer architecture designed specifically for geospatial tabular data modeling. GeoAggregators explicitly account for spatial autocorrelation and spatial heterogeneity through Gaussian-biased local attention and global positional awareness. Additionally, we introduce a new attention mechanism that uses the Cartesian product to manage the size of the model while maintaining strong expressive power. We benchmark GeoAggregator against spatial statistical models, XGBoost, and several state-of-the-art geospatial deep learning methods using both synthetic and empirical geospatial datasets. The results demonstrate that GeoAggregators achieve the best or second-best performance compared to their competitors on nearly all datasets. GeoAggregator's efficiency is underscored by its reduced model size, making it both scalable and lightweight. Moreover, ablation experiments offer insights into the effectiveness of the Gaussian bias and Cartesian attention mechanism, providing recommendations for further optimizing the GeoAggregator's performance.

地理空间Transformer轻量模型表格数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。