构建多语言句子级简化语料库，提升非英语文本可读性
Align and Shine: Building High-Quality Sentence-Aligned Corpora for Multilingual Text Simplification
- 基于平行文档自动对齐，生成跨语言句子级简化数据
- 覆盖5种语言，包含高质量对齐句对，支持多语言训练与评测
- 开源数据集，适合语言学习与无障碍阅读研究者使用
文本简化在提升各类读者（包括语言学习者和低读写能力人群）对书面信息的可访问性和理解力方面具有重要作用。尽管意义重大，目前除英语外，可用于训练和评估文本简化模型的大规模高质量数据集仍十分稀缺。本文报告了一项实验研究，通过从可比语料库中收集众包简化数据，并进行处理，构建了一个适用于多种语言（加泰罗尼亚语、英语、法语、意大利语和西班牙语）文本简化系统训练与测试的语料库。文中详细介绍了从文档级数据实现句子级对齐的机制。最终生成的对齐句对数据集已公开发布。
原文摘要 · Abstract (English)
Text simplification plays a crucial role in improving the accessibility and comprehensibility of written information for diverse audiences, including language learners and readers with limited literacy. Despite its importance, large-scale, high-quality datasets for training and evaluating text simplification models remain scarce for languages other than English. This paper reports an experimental study on the collection and processing of crowd-sourced simplification data from comparable corpora to construct a corpus suitable for both training and testing text simplification systems across multiple languages (Catalan, English, French, Italian and Spanish). We report mechanisms for sentence-level alignment from document-level data. The resulting dataset of the aligned sentence pairs is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。