arXiv:2506.12437cs.HCcs.AI2025-06被引 9

探讨情感AI如何改变人机互动,揭示其伦理与社会风险

Feeling Machines: Ethics, Culture, and the Rise of Emotional AI

  • 跨学科分析情感AI在教育、医疗等场景中的应用与影响
  • 指出情感模拟可能引发操纵、偏见及脆弱群体风险
  • 适合关注AI伦理、人机关系与政策设计的研究者

本文从批判性与跨学科视角探讨情绪响应型人工智能日益增长的影响。汇集多个领域青年研究者的观点,分析情感计算系统在教育、医疗、心理健康、照护及数字生活等领域的应用。围绕四大主题展开:情感AI的伦理问题、人机交互的文化动态、对弱势群体的风险与机遇,以及新兴的监管、设计与技术考量。作者指出,情感AI有助于提升心理福祉、促进学习与缓解孤独,但也存在情感操控、过度依赖、误判和文化偏见等风险。核心挑战包括无真实理解的共情模拟、主流社会文化规范的编码固化,以及敏感或高风险情境下保护机制不足。重点关注儿童、老年人及心理健康人群,他们可能与AI产生深层情感联结,但当前缺乏认知或法律保护。报告提出十条建议,包括透明度、认证框架、区域定制化调优、人工监督与长期研究。附录提供实用工具、模型与数据集,支持该领域后续研究。

原文摘要 · Abstract (English)

This paper explores the growing presence of emotionally responsive artificial intelligence through a critical and interdisciplinary lens. Bringing together the voices of early-career researchers from multiple fields, it explores how AI systems that simulate or interpret human emotions are reshaping our interactions in areas such as education, healthcare, mental health, caregiving, and digital life. The analysis is structured around four central themes: the ethical implications of emotional AI, the cultural dynamics of human-machine interaction, the risks and opportunities for vulnerable populations, and the emerging regulatory, design, and technical considerations. The authors highlight the potential of affective AI to support mental well-being, enhance learning, and reduce loneliness, as well as the risks of emotional manipulation, over-reliance, misrepresentation, and cultural bias. Key challenges include simulating empathy without genuine understanding, encoding dominant sociocultural norms into AI systems, and insufficient safeguards for individuals in sensitive or high-risk contexts. Special attention is given to children, elderly users, and individuals with mental health challenges, who may interact with AI in emotionally significant ways. However, there remains a lack of cognitive or legal protections which are necessary to navigate such engagements safely. The report concludes with ten recommendations, including the need for transparency, certification frameworks, region-specific fine-tuning, human oversight, and longitudinal research. A curated supplementary section provides practical tools, models, and datasets to support further work in this domain.

情感AIAI伦理人机交互社会影响

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。