SingaKids用对话式AI帮孩子学四国语言，支持图文互动和语音反馈。
SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning
- 基于多模态对话设计，结合图像描述与语音交互
- 在四语环境中提升不同水平学生的语言学习效果
- 专为儿童优化，支持多语言、趣味化教学
生成式人工智能已增强教育应用的个性化与互动性，对促进儿童语言习得具有巨大潜力。然而，在不同语言和文化背景下保持一致且稳健的表现仍具挑战，儿童友好设计需简化指令、增强互动性，并提供适龄支架以维持动机并优化学习成效。本文提出SingaKids，一种通过图片描述任务促进语言学习的对话式导师系统。该系统融合密集图像描述、多语言对话交互、语音理解与生动语音生成，在英语、中文、马来语和泰米尔语四种语言中构建沉浸式学习环境。通过多语言预训练、任务特定微调及支架优化进一步提升性能。针对小学生的实证研究显示，SingaKids能有效提供对话式教学，惠及不同表现水平的学习者。
原文摘要 · Abstract (English)
The integration of generative artificial intelligence into educational applications has enhanced personalized and interactive learning experiences, and it shows strong potential to promote young learners language acquisition. However, it is still challenging to ensure consistent and robust performance across different languages and cultural contexts, and kids-friendly design requires simplified instructions, engaging interactions, and age-appropriate scaffolding to maintain motivation and optimize learning outcomes. In this work, we introduce SingaKids, a dialogic tutor designed to facilitate language learning through picture description tasks. Our system integrates dense image captioning, multilingual dialogic interaction, speech understanding, and engaging speech generation to create an immersive learning environment in four languages: English, Mandarin, Malay, and Tamil. We further improve the system through multilingual pre-training, task-specific tuning, and scaffolding optimization. Empirical studies with elementary school students demonstrate that SingaKids provides effective dialogic teaching, benefiting learners at different performance levels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。