arXiv:2605.12243cs.CL2026-05被引 3

构建首个基于真实对话的诈骗演化预测基准,揭示大模型对诈骗进程理解仍有限。

PreScam: A Benchmark for Predicting Scam Progression from Early Conversations

论文配图:PreScam: A Benchmark for Predicting Scam Progression from Early Conversations
图 1 · 摘自论文原文
  • 从17万份报告中提炼出1.1万例结构化诈骗对话,按杀伤链分层标注
  • 模型在实时终止预测上表现优于零样本大模型,但行动预测仍不理想
  • 适合反诈研究、安全对话分析及语言模型风险评估者阅读

对话式诈骗(如恋爱和投资诈骗)正成为主要网络欺诈形式,其通过多轮对话逐步施加心理操控。现有研究多关注静态检测或合成诈骗,缺乏对真实诈骗演进过程的理解能力评估。本文提出PreScam基准,基于用户提交的177,989份原始报告,筛选并结构化为11,573个跨20类诈骗的对话实例。每个实例依据提出的诈骗杀伤链(scam kill chain)进行层级划分,并在回合层面标注诈骗者的心理行为与受害者的反应。我们在两个任务上测试模型:实时终止预测(判断对话是否接近结束)与诈骗者下一步行为预测。结果表明,监督编码器在终止预测上显著优于零样本大模型,而行动预测即使对强模型也仅达中等效果。整体显示当前模型虽能捕捉部分诈骗线索,但仍难以追踪风险升级与操控演变过程。

原文摘要 · Abstract (English)

Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lottery or unpaid toll messages, they unfold through multi-turn conversations in which scammers gradually manipulate victims using evolving psychological techniques. However, existing research mainly focuses on static scam detection or synthetic scams, leaving open whether language models can understand how real-world scams progress over time. We introduce PreScam, a benchmark for modeling scam progression from early conversations. Built from user-submitted scam reports, PreScam filters and structures 177,989 raw reports into 11,573 conversational scam instances spanning 20 scam categories. Each instance is hierarchically structured according to the scam lifecycle defined by the proposed scam kill chain, and further annotated at the turn level with scammer psychological actions and victim responses. We benchmark models on two tasks: real-time termination prediction, which estimates whether a conversation is approaching the termination stage, and scammer action prediction, which forecasts the scammer's subsequent actions. Results show a clear gap between surface-level fluency and progression modeling: supervised encoders substantially outperform zero-shot LLMs on real-time termination prediction, while next-action prediction remains only moderately successful even for strong LLMs. Taken together, these results show that current models can capture some scam-related cues, yet still struggle to track how risk escalates and how manipulation unfolds across turns.

反诈骗对话分析大模型评估行为预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。