arXiv:2410.14881cs.AIcs.CL2024-10被引 8

用检索增强生成实现实时内容审核,动态更新知识库快速应对新风险。

Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation

  • 通过检索增强生成动态调用知识库,实现分类决策的即时修正。
  • 相比微调,分类准确率更高,对对抗攻击更鲁棒,性能随知识库增大而提升。
  • 适合需要快速响应新型违规内容的AI安全团队使用。

可靠的內容審核分類器對生成式AI系統的安全至關重要。在該任務中,安全與不安全輸入之間的差異往往極其微妙,導致分類器(甚至人類)難以在缺乏上下文或解釋的情況下正確區分違規與正常樣本。通過持續模型微調來擴展風險發現與緩解也速度慢、困難且成本高,使開發者無法快速有效應對新出現的危害。我們提出一種基於檢索增強生成(Class-RAG)的分類方法。Class-RAG通過動態更新的檢索資料庫擴展其基礎大語言模型的能力,實現語義層面的即時修復,從而靈活高效地進行風險管控。實證研究表明,相比模型微調,Class-RAG在分類性能上表現更優,決策過程更具透明性,且對對抗攻擊更具魯棒性。研究還表明,Class-RAG性能隨檢索資料庫規模增加而提升,證明擴大資料庫是提升內容審核能力的一種可行且低成本途徑。

原文摘要 · Abstract (English)

Robust content moderation classifiers are essential for the safety of Generative AI systems. In this task, differences between safe and unsafe inputs are often extremely subtle, making it difficult for classifiers (and indeed, even humans) to properly distinguish violating vs. benign samples without context or explanation. Scaling risk discovery and mitigation through continuous model fine-tuning is also slow, challenging and costly, preventing developers from being able to respond quickly and effectively to emergent harms. We propose a Classification approach employing Retrieval-Augmented Generation (Class-RAG). Class-RAG extends the capability of its base LLM through access to a retrieval library which can be dynamically updated to enable semantic hotfixing for immediate, flexible risk mitigation. Compared to model fine-tuning, Class-RAG demonstrates flexibility and transparency in decision-making, outperforms on classification and is more robust against adversarial attack, as evidenced by empirical studies. Our findings also suggest that Class-RAG performance scales with retrieval library size, indicating that increasing the library size is a viable and low-cost approach to improve content moderation.

內容審核檢索增強大模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。