Maya模型提升多语言图文理解，解决低资源语言与文化偏见问题。
Maya: An Instruction Finetuned Multilingual Multimodal Model
- 基于LLaVA数据集构建8语种图文预训练数据集
- 清理毒性内容后生成无毒多语言数据集
- 支持多语言视觉语言任务，增强文化理解能力
大型视觉语言模型（VLMs）在主流语言的学术基准上表现优异，但在低资源语言和多元文化情境下的表现仍有明显差距，主要源于高质量、多样化且经过安全审核的数据稀缺。为此，我们提出开源多模态多语言模型Maya，贡献包括：1）基于LLaVA数据集构建涵盖8种语言的多语言图文预训练数据集；2）对LLaVA数据集中的毒性内容进行系统分析，并生成跨8种语言的无毒版本；3）开发支持这些语言的多语言图文模型，提升视觉语言任务中的语言与文化理解能力。代码已公开于https://github.com/nahidalam/maya。
原文摘要 · Abstract (English)
The rapid development of large Vision-Language Models (VLMs) has led to impressive results on academic benchmarks, primarily in widely spoken languages. However, significant gaps remain in the ability of current VLMs to handle low-resource languages and varied cultural contexts, largely due to a lack of high-quality, diverse, and safety-vetted data. Consequently, these models often struggle to understand low-resource languages and cultural nuances in a manner free from toxicity. To address these limitations, we introduce Maya, an open-source Multimodal Multilingual model. Our contributions are threefold: 1) a multilingual image-text pretraining dataset in eight languages, based on the LLaVA pretraining dataset; 2) a thorough analysis of toxicity within the LLaVA dataset, followed by the creation of a novel toxicity-free version across eight languages; and 3) a multilingual image-text model supporting these languages, enhancing cultural and linguistic comprehension in vision-language tasks. Code available at https://github.com/nahidalam/maya.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。