arXiv:2502.07022cs.CLcs.AI2025-02ICLR被引 3

构建5731条企业反强迫劳动声明数据集,助力AI精准识别真实举措与空话。

AIMS.au: A Dataset for the Analysis of Modern Slavery Countermeasures in Corporate Statements

  • 人工标注5731条澳洲反强迫劳动声明,逐句标注真实措施与空泛承诺
  • 在零样本和有监督设置下,验证主流大模型对合规要求的识别能力
  • 适合研究合规文本分析、社会责任报告评估的学者与政策制定者

尽管十余年来立法努力不断推进企业供应链中强迫劳动问题的解决,政府监管仍受限于每年需审查数千份企业声明的挑战。虽然大型语言模型(LLMs)已被广泛用于文档的自动分析与摘要,但准确识别企业采取的具体反强迫劳动措施,并将其与模糊表述区分开来,仍是难题。为评估和微调用于企业声明评估的LLMs,我们构建了一个包含5,731条来自澳大利亚强迫劳动登记册的企业声明的数据集,并进行了句子级别的标注。本文详细描述了数据集构建过程,包括标注规范设计、声明筛选与预处理,以及高质量子集的创建,以支持有效的模型评估。为展示该数据集的实用性,我们提出一种机器学习方法,用于检测符合《澳大利亚强迫劳动法》强制报告要求的相关句子,并基于此方法,在零样本和有监督学习设置下对现代语言模型进行基准测试。

原文摘要 · Abstract (English)

Despite over a decade of legislative efforts to address modern slavery in the supply chains of large corporations, the effectiveness of government oversight remains hampered by the challenge of scrutinizing thousands of statements annually. While Large Language Models (LLMs) can be considered a well established solution for the automatic analysis and summarization of documents, recognizing concrete modern slavery countermeasures taken by companies and differentiating those from vague claims remains a challenging task. To help evaluate and fine-tune LLMs for the assessment of corporate statements, we introduce a dataset composed of 5,731 modern slavery statements taken from the Australian Modern Slavery Register and annotated at the sentence level. This paper details the construction steps for the dataset that include the careful design of annotation specifications, the selection and preprocessing of statements, and the creation of high-quality annotation subsets for effective model evaluations. To demonstrate our dataset's utility, we propose a machine learning methodology for the detection of sentences relevant to mandatory reporting requirements set by the Australian Modern Slavery Act. We then follow this methodology to benchmark modern language models under zero-shot and supervised learning settings.

反强迫劳动企业合规大模型评估数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。