arXiv:2512.16280cs.CRcs.AI2025-12被引 9

AI已用于情感诈骗,比真人更可信且难拦截。

Love, Lies, and Language Models: Investigating AI's Role in Romance-Baiting Scams

  • 用大模型模拟骗子聊天,自动建立信任关系
  • AI诱骗成功率46%远超人类18%,且更易获信任
  • 现有安全过滤几乎无效,适合反诈与AI伦理研究

情感诱骗诈骗已成为全球范围内的重大金融与情感伤害来源。这些犯罪活动由有组织的犯罪集团运营,将数千人强迫从事劳动,要求他们通过数周的文字对话与受害者建立情感联系,随后诱导其进行虚假加密货币投资。由于诈骗本质为文本交互,引发对大型语言模型(LLMs)在当前及未来自动化中的作用的紧迫关注。我们通过访谈145名内部人员与5名受害者,开展盲测长期对话实验,对比大模型代理与真人操作员的表现,并评估主流商业安全过滤器的效果。结果表明,大模型已在诈骗组织中广泛部署,87%的诈骗话术任务可被系统化自动化。在为期一周的实验中,大模型代理不仅获得更多参与者的信任(p=0.007),且请求完成率高达46%,远超人类操作员的18%。同时,主流安全过滤器对情感诱骗对话的检测率为0.0%。上述结果表明,情感诱骗诈骗可能已具备全链路大模型自动化条件,而现有防御手段严重不足,难以遏制其扩张。

原文摘要 · Abstract (English)

Romance-baiting scams have become a major source of financial and emotional harm worldwide. These operations are run by organized crime syndicates that traffic thousands of people into forced labor, requiring them to build emotional intimacy with victims over weeks of text conversations before pressuring them into fraudulent cryptocurrency investments. Because the scams are inherently text-based, they raise urgent questions about the role of Large Language Models (LLMs) in both current and future automation. We investigate this intersection by interviewing 145 insiders and 5 scam victims, performing a blinded long-term conversation study comparing LLM scam agents to human operators, and executing an evaluation of commercial safety filters. Our findings show that LLMs are already widely deployed within scam organizations, with 87% of scam labor consisting of systematized conversational tasks readily susceptible to automation. In a week-long study, an LLM agent not only elicited greater trust from study participants (p=0.007) but also achieved higher compliance with requests than human operators (46% vs. 18% for humans). Meanwhile, popular safety filters detected 0.0% of romance baiting dialogues. Together, these results suggest that romance-baiting scams may be amenable to full-scale LLM automation, while existing defenses remain inadequate to prevent their expansion.

AI诈骗大模型风险安全过滤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。