小模型比大模型更擅长社区化内容审核
SLM-Mod: Small Language Models Surpass LLMs at Content Moderation
- 用150万条贴吧评论微调小型语言模型
- 小模型准确率高11.5%，召回率高25.7%
- 适合需要本地化审核的社区和平台
大型语言模型在内容审核等自然语言理解任务中表现良好，但实时调用成本高，且难以实现社区定制化。为此，我们探索开源小型语言模型（SLMs，参数少于150亿）在社区特定内容审核中的应用。基于15个热门Reddit社区的15万条评论，我们在零样本和少样本场景下对比了SLMs与更大规模的开源及闭源模型的表现。结果显示，SLMs在内容审核任务中超越零样本大模型，平均准确率提升11.5%，召回率提升25.7%。此外，少样本上下文学习对大模型性能提升有限，仍不及小模型。我们还展示了跨社区审核的潜力，对未来跨平台内容治理具有意义。代码与模型已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown promise in many natural language understanding tasks, including content moderation. However, these models can be expensive to query in real-time and do not allow for a community-specific approach to content moderation. To address these challenges, we explore the use of open-source small language models (SLMs) for community-specific content moderation tasks. We fine-tune and evaluate SLMs (less than 15B parameters) by comparing their performance against much larger open- and closed-sourced models in both a zero-shot and few-shot setting. Using 150K comments from 15 popular Reddit communities, we find that SLMs outperform zero-shot LLMs at content moderation -- 11.5% higher accuracy and 25.7% higher recall on average across all communities. Moreover, few-shot in-context learning leads to only a marginal increase in the performance of LLMs, still lacking compared to SLMs. We further show the promise of cross-community content moderation, which has implications for new communities and the development of cross-platform moderation techniques. Finally, we outline directions for future work on language model based content moderation. Code and models can be found at https://github.com/AGoyal0512/SLM-Mod.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。