用RWKV模型提升内容审核效率，打造可压缩的小型化审核系统。
Experimentation in Content Moderation using RWKV
- 基于多模态数据构建专用蒸馏数据集，支持小型化模型训练。
- 用55万条文本与8万张图像响应数据微调RWKV，实现高效准确审核。
- 适合关注轻量化部署与资源受限场景的内容安全研究者。
本文通过针对性实验探究了RWKV模型在内容审核中的有效性。我们引入了一个专为小模型蒸馏设计的新数据集，涵盖呈现社会挑战的图文音视频数据。借助先进大语言模型，生成了558,958条文本和83,625条图像的标注响应，用于训练与优化内容审核系统。核心实验聚焦于微调RWKV模型，利用其高效的CPU架构应对大规模内容审核任务。本研究不仅展示了该数据集在知识蒸馏中的潜力，也验证了RWKV在提升审核系统准确性与效率方面的可行性，为开发更紧凑、资源节约型模型提供了新路径。数据集与模型可在HuggingFace获取：https://huggingface.co/modrwkv
原文摘要 · Abstract (English)
This paper investigates the RWKV model's efficacy in content moderation through targeted experimentation. We introduce a novel dataset specifically designed for distillation into smaller models, enhancing content moderation practices. This comprehensive dataset encompasses images, videos, sounds, and text data that present societal challenges. Leveraging advanced Large Language Models (LLMs), we generated an extensive set of responses -- 558,958 for text and 83,625 for images -- to train and refine content moderation systems. Our core experimentation involved fine-tuning the RWKV model, capitalizing on its CPU-efficient architecture to address large-scale content moderation tasks. By highlighting the dataset's potential for knowledge distillation, this study not only demonstrates RWKV's capability in improving the accuracy and efficiency of content moderation systems but also paves the way for developing more compact, resource-efficient models in this domain. Datasets and models can be found in HuggingFace: https://huggingface.co/modrwkv
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。