构建印度36种语言的翻译生态，解决多语种、多文字难题。
BhashaVerse : Translation Ecosystem for Indian Subcontinent Languages
- 整合现有资源与合成数据，构建跨36语言的平行语料库。
- 覆盖多语言对与多脚本，支持低资源语言如卡西语、桑塔利语。
- 适合关注南亚多语言翻译与本土化应用的研究者与开发者。
本文致力于为印度次大陆36种语言（包括阿萨姆语、阿瓦德语、孟加拉语、比约普里语、布拉吉语、博多语、多格里语、英语、果尔迪语、古吉拉特语、印地语、印地英混合语、霍语、卡纳达语、康格里语、克什米尔语（阿拉伯与天城文）、卡西语、米佐语、马加希语、迈蒂利语、马拉雅拉姆语、马拉地语、曼尼普里语（孟加拉与梅泰文）、尼泊尔语、奥里亚语、旁遮普语、梵语、桑塔利语、僧伽罗语、信德语（阿拉伯与天城文）、泰米尔语、图鲁语、泰卢固语、乌尔都语）开发机器翻译模型及相关应用。实现此目标需构建所有36×36语言对的平行语料及其他类型语料，应对脚本差异、发音差异与句法多样性等挑战。例如，克什米尔语与信德语使用多种脚本，需进行脚本归一化以实现对齐；而卡西语、桑塔利语等低资源语言则需通过合成数据增强以确保覆盖范围与质量。为此，本工作提出策略：利用现有资源、构建平行数据集、生成领域专用语料、采用合成数据技术。此外，从标准与话语层面翻译、领域特定翻译、基于参考与无参考评估、错误分析及自动后编辑等多个维度评估机器翻译性能。通过整合上述要素，研究建立了一个全面框架，旨在提升机器翻译质量，促进印度语言多样生态中的跨语言交流。
原文摘要 · Abstract (English)
This paper focuses on developing translation models and related applications for 36 Indian languages, including Assamese, Awadhi, Bengali, Bhojpuri, Braj, Bodo, Dogri, English, Konkani, Gondi, Gujarati, Hindi, Hinglish, Ho, Kannada, Kangri, Kashmiri (Arabic and Devanagari), Khasi, Mizo, Magahi, Maithili, Malayalam, Marathi, Manipuri (Bengali and Meitei), Nepali, Oriya, Punjabi, Sanskrit, Santali, Sinhala, Sindhi (Arabic and Devanagari), Tamil, Tulu, Telugu, and Urdu. Achieving this requires parallel and other types of corpora for all 36 * 36 language pairs, addressing challenges like script variations, phonetic differences, and syntactic diversity. For instance, languages like Kashmiri and Sindhi, which use multiple scripts, demand script normalization for alignment, while low-resource languages such as Khasi and Santali require synthetic data augmentation to ensure sufficient coverage and quality. To address these challenges, this work proposes strategies for corpus creation by leveraging existing resources, developing parallel datasets, generating domain-specific corpora, and utilizing synthetic data techniques. Additionally, it evaluates machine translation across various dimensions, including standard and discourse-level translation, domain-specific translation, reference-based and reference-free evaluation, error analysis, and automatic post-editing. By integrating these elements, the study establishes a comprehensive framework to improve machine translation quality and enable better cross-lingual communication in India's linguistically diverse ecosystem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。