通过动态阈值与机器学习，高效识别高相似性地理簇
Efficient Identification of High Similarity Clusters in Polygon Datasets
- 用核密度估计动态调优相似性阈值
- 减少90%以上需验证的簇，计算量显著降低
- 适合处理大规模地理数据的科研与应用
Shapely 2.0和Triton等工具虽能提升空间相似性计算效率，但在极大规模数据下仍面临计算负担过重问题。为此，我们提出一种框架，通过动态相似性阈值调整、监督调度与召回约束优化，减少需验证的簇数量，从而降低系统负载。该框架结合核密度估计（KDE）动态确定相似性阈值，并利用机器学习模型优先处理高潜力簇，实现计算成本大幅下降而精度不减。实验表明，该方法在保持用户定义的精度与召回率前提下，具备良好的可扩展性和有效性，为大规模地理空间分析提供了实用解决方案。
原文摘要 · Abstract (English)
Advancements in tools like Shapely 2.0 and Triton can significantly improve the efficiency of spatial similarity computations by enabling faster and more scalable geometric operations. However, for extremely large datasets, these optimizations may face challenges due to the sheer volume of computations required. To address this, we propose a framework that reduces the number of clusters requiring verification, thereby decreasing the computational load on these systems. The framework integrates dynamic similarity index thresholding, supervised scheduling, and recall-constrained optimization to efficiently identify clusters with the highest spatial similarity while meeting user-defined precision and recall requirements. By leveraging Kernel Density Estimation (KDE) to dynamically determine similarity thresholds and machine learning models to prioritize clusters, our approach achieves substantial reductions in computational cost without sacrificing accuracy. Experimental results demonstrate the scalability and effectiveness of the method, offering a practical solution for large-scale geospatial analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。