arXiv:2504.04855cs.AI2025-04被引 4

用AI代理自动检测结构化数据偏见,支持定制化分析

BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agents

  • 设计多智能体协同框架,按用户需求分阶段执行偏见检测
  • 在多个测试用例上表现优异,提供带解释与可视化的详细结果
  • 首次构建标准化评估基准,推动数据偏见检测研究发展

检测结构化数据中的偏见是一项复杂且耗时的任务。现有自动化方法在数据类型多样性上受限,且高度依赖人工逐案处理,缺乏通用性。当前大语言模型(LLM)代理在数据科学领域已取得显著进展,但其在数据偏见检测方面的潜力尚未充分探索。为此,我们提出首个端到端、多智能体协同的框架BIASINSPECTOR,基于用户特定需求实现结构化数据偏见的自动化检测。该框架首先制定多阶段分析计划,再调用多样化且适配的工具集执行任务,输出包含解释与可视化在内的详细结果。为解决评估标准缺失问题,我们进一步构建了一个包含多种评估指标和大量测试用例的综合性基准。大量实验表明,本框架在结构化数据偏见检测中表现出色,为更公平的数据应用树立了新标杆。

原文摘要 · Abstract (English)

Detecting biases in structured data is a complex and time-consuming task. Existing automated techniques are limited in diversity of data types and heavily reliant on human case-by-case handling, resulting in a lack of generalizability. Currently, large language model (LLM)-based agents have made significant progress in data science, but their ability to detect data biases is still insufficiently explored. To address this gap, we introduce the first end-to-end, multi-agent synergy framework, BIASINSPECTOR, designed for automatic bias detection in structured data based on specific user requirements. It first develops a multi-stage plan to analyze user-specified bias detection tasks and then implements it with a diverse and well-suited set of tools. It delivers detailed results that include explanations and visualizations. To address the lack of a standardized framework for evaluating the capability of LLM agents to detect biases in data, we further propose a comprehensive benchmark that includes multiple evaluation metrics and a large set of test cases. Extensive experiments demonstrate that our framework achieves exceptional overall performance in structured data bias detection, setting a new milestone for fairer data applications.

偏见检测多智能体大模型应用数据公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。