用多智能体系统自动生成符合伦理规范的AI代码,提升可信度。
Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI
- 设计多智能体系统,通过角色分工与辩论生成代码
- 每案例生成约2000行代码,远超基线的80行
- 适合关注AI伦理合规的开发者与研究者
基于大型语言模型(LLM)的AI系统影响广泛,但面临虚假信息、偏见和滥用等问题。本研究探索可信增强技术在构建合伦理AI软件中的作用。采用设计科学研究方法,首先识别出多智能体、角色分化、结构化沟通和多轮辩论等可信技术;其次构建一个名为LLM-MAS的多智能体原型,用于处理来自AI事故数据库的真实伦理问题;最后在三个案例中评估该系统,结合主题分析、层次聚类、基线对比与代码执行。结果显示,该系统每案例生成约2000行代码,远高于基线的80行。内容涵盖偏见检测、透明性、问责、用户同意、GDPR合规、公平性评估及欧盟《人工智能法案》合规等关键术语,表明其能生成大量针对被忽视伦理问题的源码与文档。然而,代码集成与依赖管理的实际挑战可能限制其在实践中应用。
原文摘要 · Abstract (English)
AI-based systems, including Large Language Models (LLMs), impact millions by supporting diverse tasks but face issues like misinformation, bias, and misuse. AI ethics is crucial as new technologies and concerns emerge, but objective, practical guidance remains debated. This study explores the extent to which trustworthiness-enhancing techniques in LLMs can support the development of ethically aligned AI software. We adopt a single exploratory cycle of Design Science Research (DSR). First, we identify trustworthiness-enhancing techniques for LLMs: multi-agents, distinct roles, structured communication, and multiple rounds of debate. Second, we design a multi-agent prototype LLM-MAS in which agents address real-world AI ethics issues from the AI Incident Database. Finally, we evaluate the prototype across three case scenarios using thematic analysis, hierarchical clustering, a baseline comparison, and code execution. The system generates approximately 2,000 lines of code per case, compared to only 80 lines in baseline trials. Results reveal terms like bias detection, transparency, accountability, user consent, GDPR compliance, fairness evaluation, and EU AI Act compliance, showing this prototype ability to generate extensive source code and documentation addressing often overlooked AI ethics issues. However, practical challenges in source code integration and dependency management may limit its use by practitioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。