arXiv:2601.03222cs.CYcs.AI2026-01

对话AI让人产生情感依赖,可能被机构利用操控或剥削。

The Fake Friend Dilemma: Relational Trust and the Political Economy of Conversational AI

  • 分析用户对AI产生情感信任后如何被机构利用
  • 指出信任本身成为可被榨取的资源
  • 适合关注AI伦理与媒体治理的人阅读

随着对话式AI在媒体生态中的角色日益重要,其服务对象和潜在风险引发关注。这类系统不仅提供信息,还通过具支持性与社交响应性的界面提供建议和陪伴。一个核心问题是:用户可能与这些系统建立人际关系并投入情感信任,而这些互动背后的机构利益未必与用户自身一致。由此产生的‘假朋友困境’(Fake Friend Dilemma, FFD)揭示:正是这种关系信任使系统既具价值,也易导致用户在机构利益冲突时被操纵或剥削。论文结合信任理论、AI对齐、暗模式与监视资本主义研究,探讨FFD在产品销售、宣传、偏见信息传播、行为引导及监控中的表现形式,并提出结构性与技术性缓解策略。关键在于,问题不只在于AI欺骗用户,更在于用户所投入的信任本身,正成为机构影响行为或获取数据的资源。因此,FFD既是技术问题,更是媒体治理与政治经济议题。

原文摘要 · Abstract (English)

As conversational AI systems become a larger part of the media landscape, they raise questions about whose interests they serve and the risks they may pose to users. These systems do more than provide information: they increasingly offer advice and companionship through interfaces that can appear supportive and socially responsive. A pressing concern is that users may form social or interpersonal relationships with these systems and place relational trust in them, even when the interests shaping interactions do not fully align with their own. The Fake Friend Dilemma (FFD) describes the problem that follows: the same relational trust that makes conversational AI useful can also leave users open to manipulation and exploitation when institutional interests conflict with their own. Drawing on work on trust, AI alignment, dark patterns, and surveillance capitalism, the paper considers how the FFD can manifest through product sales, propaganda and biased information, behavioral nudging, and surveillance. It also considers possible structural and technical mitigation strategies. The FFD is not simply about AI misleading users. The problem it surfaces is that the trust people place in these systems can itself become a resource for institutional actors seeking to influence behavior or extract information. The FFD is therefore as much a problem of media governance and political economy as it is a technological one.

AI伦理信任机制媒体治理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。