arXiv:2505.07852cs.CLcs.AI2025-05被引 3

用LLM区分聊天中的欺诈与正常话题切换,提升实时反诈准确率。

Joint Detection of Fraud and Concept Drift inOnline Conversations with LLM-Assisted Judgment

  • 先用集成模型找可疑对话,再用单类漂移检测识别话题突变
  • 检测到漂移时调用LLM判断是诈骗还是正常换话题,误报率降低
  • 适合需要高可解释性的在线平台反欺诈系统部署

数字通信平台中的虚假互动检测仍具挑战且研究不足。这些行为可能表现为无害垃圾信息,也可能演变为复杂诈骗,难以早期识别恶意意图。传统方法依赖静态异常检测,无法适应动态对话变化。主要缺陷在于将良性话题转换(概念漂移)误判为欺诈,导致误报或漏报。本文提出两阶段检测框架:首先使用定制集成分类模型识别可疑对话;为提高可靠性,引入单类漂移检测器(OCDD)分析已标记对话中的话题变化。若检测到漂移,则调用大语言模型(LLM)判断该变化是欺诈操纵还是合法话题转换;若未发现漂移,则推断为类似垃圾信息的行为。在社交工程聊天场景数据集上验证,本框架在实时欺诈检测中显著提升准确率与可解释性。为评估权衡,与使用双LLM的基线方法对比,结果表明本模块化设计更具优势。

原文摘要 · Abstract (English)

Detecting fake interactions in digital communication platforms remains a challenging and insufficiently addressed problem. These interactions may appear as harmless spam or escalate into sophisticated scam attempts, making it difficult to flag malicious intent early. Traditional detection methods often rely on static anomaly detection techniques that fail to adapt to dynamic conversational shifts. One key limitation is the misinterpretation of benign topic transitions referred to as concept drift as fraudulent behavior, leading to either false alarms or missed threats. We propose a two stage detection framework that first identifies suspicious conversations using a tailored ensemble classification model. To improve the reliability of detection, we incorporate a concept drift analysis step using a One Class Drift Detector (OCDD) to isolate conversational shifts within flagged dialogues. When drift is detected, a large language model (LLM) assesses whether the shift indicates fraudulent manipulation or a legitimate topic change. In cases where no drift is found, the behavior is inferred to be spam like. We validate our framework using a dataset of social engineering chat scenarios and demonstrate its practical advantages in improving both accuracy and interpretability for real time fraud detection. To contextualize the trade offs, we compare our modular approach against a Dual LLM baseline that performs detection and judgment using different language models.

欺诈检测概念漂移LLM应用对话分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。