用记忆化摘要的智能体系统,识别长期对话骗局并解释判断依据。
An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory

- 构建基于摘要记忆的智能体系统,追踪跨多轮对话的信任建立过程。
- 在公开数据集上实现97.8%准确率,全量检测出83个骗局案例。
- 用户研究证实系统提升信任感与使用信心,易用性达标。
随着生成式人工智能的快速发展,对话式诈骗威胁日益严重。此类诈骗常持续数周甚至数月,逐步建立信任后诱导转账或泄露敏感信息。现有检测系统多聚焦单条消息,难以应对这种渐进式攻击。本文提出一种可解释的智能体系统,用于识别复杂对话骗局,并发布首个包含八类骗局的公开基准ConScamBench-278,支持可复现评估与扩展。单消息检测器召回率达100%;对话级检测器在LoveFraud02语料库中成功识别全部83个骗局,且在ConScamBench-278上达到97.8%准确率(95%置信区间[95.4, 99.0])。两个用户研究(N=100和N=45)显示,参与者普遍对可疑对话感到困惑;在非控制预/后对比中,用户自评信任度、自信度及对AI检测的需求均显著上升(p<0.001,Wilcoxon符号秩检验)。系统可用性量表得分为74.7(95% CI [72.5, 76.9]),高于行业基准。
原文摘要 · Abstract (English)
Following the rapid progress of generative Artificial Intelligence, there is a growing threat posed by conversational scams. These scams often span over multiple weeks or months, gradually build trust and request for money or sensitive information. Existing scam-detection systems mainly focus on isolated messages, which renders them inadequate against this evolving threat. This paper extends single-message phishing detection and presents an explainable agentic system for detecting sophisticated conversational scams. It also introduces ConScamBench-278, an initial public multi-category benchmark for conversational scam detection spanning eight scam types, released to support reproducible evaluation and future expansion. On isolated messages the single-message detector attains 100% phishing recall, while the conversation-level detector identifies all conversational scams in the public LoveFraud02 corpus (83/83) and reaches 97.8% accuracy (95% CI [95.4, 99.0]) on ConScamBench-278. Two user studies (N = 100 and N = 45) further motivate the system: participants report frequently experiencing uncertainty when judging suspicious conversations. In an uncontrolled pre/post comparison, users self-reported trust, self-confidence, and perceived need for AI-based scam detection all increased (p < 0.001, Wilcoxon signed-rank). The system also receives a System Usability Scale score of 74.7 (95% CI [72.5, 76.9]), above the established usability benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。