为开源模型训练数据设计去毒流程，提升安全性。
Toxicity of the Commons: Curating Open-Source Pre-Training Data
- 构建专用去毒数据集ToxicCommons，覆盖五类歧视与暴力内容。
- 训练出可高效检测开放数据毒性的分类器Celadon。
- 平衡过滤程度与训练数据量，适合关注安全的开发者使用。
开源大语言模型日益普及，但其训练数据仍缺乏开放实践。尽管研究者致力于提升模型安全性，现有毒性过滤方法难以适用于公开领域文本，因这些数据多为历史文档经光学字符识别生成，形式与内容异于普通网页文本。本文提出一种完全开源的去毒数据清洗流程：构建包含五类歧视（种族/出身、性别/性取向、宗教、残障、暴力）标注的ToxicCommons数据集；基于此训练自定义分类器Celadon，实现对开放数据中毒性内容的高效大规模检测；并提出兼顾安全与数据可用性的平衡过滤策略。
原文摘要 · Abstract (English)
Open-source large language models are becoming increasingly available and popular among researchers and practitioners. While significant progress has been made on open-weight models, open training data is a practice yet to be adopted by the leading open-weight models creators. At the same time, there researchers are working to make language models safer. We propose a data curation pipeline to reduce harmful outputs by models trained on public domain data. There are unique challenges to working with public domain data, as these sources differ from web text in both form and content. Many sources are historical documents and are the result of Optical Character Recognition (OCR). Consequently, current state-of-the-art approaches to toxicity filtering are often infeasible or inappropriate for open data models. In this paper, we introduce a new fully open-source pipeline for open-data toxicity filtering. Our contributions are threefold. We create a custom training dataset, ToxicCommons, which is composed of texts which have been classified across five different dimensions (racial/origin-based, gender/sex-based, religious, ability-based discrimination, and violence). We use this dataset to train a custom classifier, Celadon, that can be used to detect toxic content in open data more efficiently at a larger scale. Finally, we describe the balanced approach to content filtration that optimizes safety filtering with respect to the filtered data available for training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。