用全球新闻数据微调大模型，补全AI影响评估的盲区
Towards Leveraging News Media to Support Impact Assessment of AI Technologies
- 用266个媒体来源的新闻微调开源大模型，挖掘真实社会影响
- 微调后的Mistral-7B比GPT-4覆盖更多元的负面影响类别
- 为政策制定者和研究者提供更全面的AI社会影响分析工具
现有专家驱动的影响评估框架可能忽略AI对公众行为、政策及文化地理背景的影响。本研究利用来自全球30个国家266个新闻领域的多样化文章，对开源大模型进行微调，以增强影响评估的多样性。研究发现：（1）微调后的开源大模型在连贯性、结构、相关性和可信度四个定性维度上可生成高质量负面效应；（2）仅用小规模开源模型Mistral-7B，在新闻数据上微调后，能覆盖GPT-4遗漏的更多影响类别，展现出更强的广度与实用性。
原文摘要 · Abstract (English)
Expert-driven frameworks for impact assessments (IAs) may inadvertently overlook the effects of AI technologies on the public's social behavior, policy, and the cultural and geographical contexts shaping the perception of AI and the impacts around its use. This research explores the potentials of fine-tuning LLMs on negative impacts of AI reported in a diverse sample of articles from 266 news domains spanning 30 countries around the world to incorporate more diversity into IAs. Our findings highlight (1) the potential of fine-tuned open-source LLMs in supporting IA of AI technologies by generating high-quality negative impacts across four qualitative dimensions: coherence, structure, relevance, and plausibility, and (2) the efficacy of small open-source LLM (Mistral-7B) fine-tuned on impacts from news media in capturing a wider range of categories of impacts that GPT-4 had gaps in covering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。