超小模型实现高精度毒害内容检测，适合低资源部署。
Tiny-Toxic-Detector: A compact transformer-based model for toxic content detection
- 4层轻量Transformer，仅210万参数，专注毒害内容识别。
- 在ToxiGen上达90.97%准确率，Jigsaw上86.98%，媲美大模型。
- 适合社交平台实时监控，兼顾性能与计算效率。
本文提出Tiny-toxic-detector，一种基于Transformer的紧凑型毒害内容检测模型。尽管参数量仅有210万，该模型在基准数据集上表现优异：ToxiGen上准确率达90.97%，Jigsaw上达86.98%，性能可比超过其50倍大小的模型。模型采用4个编码器层，每层含2个注意力头，嵌入维度64，前馈网络维度128。在公开与私有数据集上训练后，展现出高效任务专用模型在应对网络毒性方面的潜力。论文涵盖架构设计、训练流程、性能评测及局限性，强调其在社交媒体监控与内容审核中的适用性。通过在显著降低计算开销的前提下实现高精度，该模型推动了可持续、可扩展的AI内容治理发展。
原文摘要 · Abstract (English)
This paper presents Tiny-toxic-detector, a compact transformer-based model designed for toxic content detection. Despite having only 2.1 million parameters, Tiny-toxic-detector achieves competitive performance on benchmark datasets, with 90.97% accuracy on ToxiGen and 86.98% accuracy on the Jigsaw dataset, rivaling models over 50 times its size. This efficiency enables deployment in resource-constrained environments, addressing the need for effective content moderation tools that balance performance with computational efficiency. The model architecture features 4 transformer encoder layers, each with 2 attention heads, an embedding dimension of 64, and a feedforward dimension of 128. Trained on both public and private datasets, Tiny-toxic-detector demonstrates the potential of efficient, task-specific models for addressing online toxicity. The paper covers the model architecture, training process, performance benchmarks, and limitations, underscoring its suitability for applications such as social media monitoring and content moderation. By achieving results comparable to much larger models while significantly reducing computational demands, Tiny-toxic-detector represents progress toward more sustainable and scalable AI-driven content moderation solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。