提出可量化地理偏见的信息论框架,支持跨模型公平比较。
GeoBS: Information-Theoretic Quantification of Geographic Bias in AI Models
- 基于信息论构建通用地理偏见评估框架。
- 在8个数据集、3项任务中发现模型普遍存在地理偏见。
- 考虑多尺度、距离衰减等空间特性,适合空间公平研究者使用。
AI模型(尤其是基础模型)的广泛应用带来了深远影响,但也引发显著伦理问题,如偏见。尽管已有大量工作关注社会偏见的量化与缓解,地理偏见(geo-bias)却未受足够重视,且现有度量方法存在模型依赖性或空间隐含性。本文提出一种信息论框架GeoBS(Geo-Bias Scores),实现模型无关、普遍适用且空间显式的地理偏见评估。通过该框架,我们解释并分析了现有度量方法,并提出三种新指标,分别考虑多尺度、距离衰减和各向异性等复杂空间因素。在3项任务、8个数据集和8个模型上进行广泛实验,结果表明任务特定的GeoAI模型和通用基础模型均存在多种地理偏见。该框架不仅深化对地理偏见的技术理解,也为将空间公平融入AI系统设计、部署与评估奠定基础。
原文摘要 · Abstract (English)
The widespread adoption of AI models, especially foundation models (FMs), has made a profound impact on numerous domains. However, it also raises significant ethical concerns, including bias issues. Although numerous efforts have been made to quantify and mitigate social bias in AI models, geographic bias (in short, geo-bias) receives much less attention, which presents unique challenges. While previous work has explored ways to quantify geo-bias, these measures are model-specific (e.g., mean absolute deviation of LLM ratings) or spatially implicit (e.g., average fairness scores of all spatial partitions). We lack a model-agnostic, universally applicable, and spatially explicit geo-bias evaluation framework that allows researchers to fairly compare the geo-bias of different AI models and to understand what spatial factors contribute to the geo-bias. In this paper, we establish an information-theoretic framework for geo-bias evaluation, called GeoBS (Geo-Bias Scores). We demonstrate the generalizability of the proposed framework by showing how to interpret and analyze existing geo-bias measures under this framework. Then, we propose three novel geo-bias scores that explicitly take intricate spatial factors (multi-scalability, distance decay, and anisotropy) into consideration. Finally, we conduct extensive experiments on 3 tasks, 8 datasets, and 8 models to demonstrate that both task-specific GeoAI models and general-purpose foundation models may suffer from various types of geo-bias. This framework will not only advance the technical understanding of geographic bias but will also establish a foundation for integrating spatial fairness into the design, deployment, and evaluation of AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。