用信息抽取技术挖掘影响人类信任AI的因素，构建首个相关标注数据集。
Can AI Extract Antecedent Factors of Human Trust in AI? An Application of Information Extraction for Scientific Literature in Behavioural and Computer Sciences
- 基于领域专家设计标注规范，构建首个该领域英文标注数据集。
- 发现当前提示工程式大模型无法有效完成此任务，需监督学习。
- 适用于人机信任、AI伦理研究者，助力可解释性分析。
从科学文献中提取信息是将文本中的非结构化知识转化为结构化数据的关键技术，可用于下游决策任务。在人工智能信任研究中，影响人类对AI应用信任的因素关系复杂。本文从信息抽取视角出发，结合领域专家输入，精心设计标注指南，创建了首个该领域的英文标注数据集，探索大模型引导的标注方法，并使用大语言模型在命名实体和关系抽取任务上进行基准测试。结果表明，该问题需要监督学习，而当前基于提示的大模型方法尚不可行。
原文摘要 · Abstract (English)
Information extraction from the scientific literature is one of the main techniques to transform unstructured knowledge hidden in the text into structured data which can then be used for decision-making in down-stream tasks. One such area is Trust in AI, where factors contributing to human trust in artificial intelligence applications are studied. The relationships of these factors with human trust in such applications are complex. We hence explore this space from the lens of information extraction where, with the input of domain experts, we carefully design annotation guidelines, create the first annotated English dataset in this domain, investigate an LLM-guided annotation, and benchmark it with state-of-the-art methods using large language models in named entity and relation extraction. Our results indicate that this problem requires supervised learning which may not be currently feasible with prompt-based LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。