用语言模型与图神经网络结合,识别伪造的AI生成差评。
Detecting LLM-Generated Spam Reviews by Integrating Language Model Embeddings and Graph Neural Network
- 融合预训练语言模型嵌入与门控图变换器,捕捉文本与行为特征。
- 在三个AI生成差评数据集上,精度和召回率分别提升44.22%和43.01%。
- 模型轻量且少需标注数据,适合真实平台快速部署。
大语言模型(LLMs)的兴起使得生成高度逼真的伪造差评成为可能,这些评论极似真人撰写,严重威胁在线平台的可信度。本文首先使用三种不同LLM,基于商品元数据和真实参考评论构建了三组真实的AI生成差评数据集,GPT-4.1评估确认其具有高说服力和欺骗性。为应对该挑战,我们提出FraudSquad——一种混合检测模型,将预训练语言模型的文本嵌入与门控图变压器结合,实现对垃圾评论节点的分类。该模型无需人工特征工程或大量训练资源,即可同时捕捉语义与行为信号。实验表明,在三个AI生成差评数据集上,FraudSquad的精度和召回率分别比现有最优方法提升最高达44.22%和43.01%,并在两个真人撰写差评数据集上也表现优异。此外,模型规模小、所需标注数据极少,具备实际应用潜力。贡献包括新合成数据集、实用检测框架及对应对LLM时代垃圾评论的紧迫性实证。代码与数据已公开于:https://anonymous.4open.science/r/FraudSquad-5389/。
原文摘要 · Abstract (English)
The rise of large language models (LLMs) has enabled the generation of highly persuasive spam reviews that closely mimic human writing. These reviews pose significant challenges for existing detection systems and threaten the credibility of online platforms. In this work, we first create three realistic LLM-generated spam review datasets using three distinct LLMs, each guided by product metadata and genuine reference reviews. Evaluations by GPT-4.1 confirm the high persuasion and deceptive potential of these reviews. To address this threat, we propose FraudSquad, a hybrid detection model that integrates text embeddings from a pre-trained language model with a gated graph transformer for spam node classification. FraudSquad captures both semantic and behavioral signals without relying on manual feature engineering or massive training resources. Experiments show that FraudSquad outperforms state-of-the-art baselines by up to 44.22% in precision and 43.01% in recall on three LLM-generated datasets, while also achieving promising results on two human-written spam datasets. Furthermore, FraudSquad maintains a modest model size and requires minimal labeled training data, making it a practical solution for real-world applications. Our contributions include new synthetic datasets, a practical detection framework, and empirical evidence highlighting the urgency of adapting spam detection to the LLM era. Our code and datasets are available at: https://anonymous.4open.science/r/FraudSquad-5389/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。