arXiv:2511.05532cs.CL2025-11

用提示学习实现可定制的有害内容检测,用户无需重训练就能自定义过滤规则。

Beyond One-Size-Fits-All: Personalized Harmful Content Detection with In-Context Learning

  • 基于大模型的上下文学习,通过简单提示调整检测策略。
  • 仅需一个示例或定义即可实现有效个性化,性能接近专用模型。
  • 适合关注隐私、需要灵活调节内容过滤的用户和研究者。

有害网络内容(如侮辱性言论、垃圾信息、负面情绪)的泛滥亟需强大且灵活的审核系统。现有系统多为集中式、任务专用,透明度低且忽视用户偏好,难以适应隐私敏感或去中心化环境。本文提出一种新框架,利用大模型的上下文学习(ICL)统一处理毒性、垃圾信息和负面情绪的二分类、多分类及多标签检测。关键优势在于轻量级个性化:用户可通过简单提示新增屏蔽类别、解除屏蔽或扩展对语义变体的识别,无需模型重训。在TextDetox、UCI SMS、SST2等公开基准及新标注的Mastodon数据集上的实验表明:(i) 大模型具备强跨任务泛化能力,常达到甚至超过专用微调模型性能;(ii) 仅需一个用户提供的示例或定义即可实现有效个性化;(iii) 在提示中加入标签定义或推理理由显著提升对噪声真实数据的鲁棒性。本工作标志着从“一刀切”审核向用户中心、隐私保护、高度可适配的内容安全系统的范式转变。为促进复现与未来研究,代码已开源于GitHub,Mastodon数据集发布于Hugging Face。

原文摘要 · Abstract (English)

The proliferation of harmful online content--e.g., toxicity, spam, and negative sentiment--demands robust and adaptable moderation systems. However, prevailing moderation systems are centralized and task-specific, offering limited transparency and neglecting diverse user preferences--an approach ill-suited for privacy-sensitive or decentralized environments. We propose a novel framework that leverages in-context learning (ICL) with foundation models to unify the detection of toxicity, spam, and negative sentiment across binary, multi-class, and multi-label settings. Crucially, our approach enables lightweight personalization, allowing users to easily block new categories, unblock existing ones, or extend detection to semantic variations through simple prompt-based interventions--all without model retraining. Extensive experiments on public benchmarks (TextDetox, UCI SMS, SST2) and a new, annotated Mastodon dataset reveal that: (i) foundation models achieve strong cross-task generalization, often matching or surpassing task-specific fine-tuned models; (ii) effective personalization is achievable with as few as one user-provided example or definition; and (iii) augmenting prompts with label definitions or rationales significantly enhances robustness to noisy, real-world data. Our work demonstrates a definitive shift beyond one-size-fits-all moderation, establishing ICL as a practical, privacy-preserving, and highly adaptable pathway for the next generation of user-centric content safety systems. To foster reproducibility and facilitate future research, we publicly release our code on GitHub and the annotated Mastodon dataset on Hugging Face.

内容安全上下文学习个性化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。