用大模型生成对抗样本，测试社交媒体内容审核的鲁棒性。
Attacking Misinformation Detection Using Adversarial Examples Generated by Language Models
- 用大模型改写文本，再逐步微调以骗过分类器。
- 在查询次数受限下，对长文本攻击成功率超80%。
- 适合研究内容审核安全或对抗攻防的学者。
大型语言模型在诸多应用中表现优异，但能否被用来攻击社交媒体平台的内容过滤算法？本文研究生成对抗样本以测试文本分类模型在识别低可信度内容（如宣传、虚假信息、谣言、极端化新闻）时的鲁棒性。我们模拟内容审核场景，设定攻击者查询次数上限。提出TREPAT方法：先用大模型根据语义保持类任务（如文本简化、风格迁移）提示生成初始改写，再通过束搜索将修改分解为小步调整，直至目标分类器误判。评估包括：(1) 在多种提示、模型和查询限制下的定量测试；(2) 对生成文本的定向人工评估；(3) 语言学定性分析。结果表明，在受限条件下本方法显著优于现有方案，尤其在长文本（如新闻文章）场景中，当穷举搜索不可行时仍具高攻击成功率。
原文摘要 · Abstract (English)
Large language models have many beneficial applications, but can they also be used to attack content-filtering algorithms in social media platforms? We investigate the challenge of generating adversarial examples to test the robustness of text classification algorithms detecting low-credibility content, including propaganda, false claims, rumours and hyperpartisan news. We focus on simulation of content moderation by setting realistic limits on the number of queries an attacker is allowed to attempt. Within our solution (TREPAT), initial rephrasings are generated by large language models with prompts inspired by meaning-preserving NLP tasks, such as text simplification and style transfer. Subsequently, these modifications are decomposed into small changes, applied through beam search procedure, until the victim classifier changes its decision. We perform (1) quantitative evaluation using various prompts, models and query limits, (2) targeted manual assessment of the generated text and (3) qualitative linguistic analysis. The results confirm the superiority of our approach in the constrained scenario, especially in case of long input text (news articles), where exhaustive search is not feasible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。