arXiv:2604.01941cs.CVcs.AI2026-04

为幼儿教育图像生成精准描述,构建了首个大规模专业标注数据集。

Captioning Daily Activity Images in Early Childhood Education: Benchmark and Algorithm

  • 提出混合训练框架RSRS,动态切换强化学习与监督微调。
  • 在新数据集上实现51.06的教师玩具识别得分,显著领先基线。
  • 适合教育科技、智能评估等专业场景应用。

针对幼儿教育图像描述任务,现有方法受限于缺乏大规模领域专用数据集,且传统训练范式难以提升专业物体命名能力。为此,本文构建了ECAC——一个包含256,121张真实场景图像的大规模基准数据集,配有专家级描述与细粒度标签,并设计了教学玩具识别分数(TTS)评估协议以衡量专业物体命名准确性。同时提出RSRS(奖励条件切换的强化学习与监督微调)框架,通过将零奖励难样本重定向至监督微调,缓解优势崩溃问题,实现稳定优化。基于此,我们开发了面向该领域的多模态大模型KinderMM-Cap-3B。实验表明,该模型在TTS上达到51.06,显著优于现有最佳方法,同时保持高质量生成效果,展现出在教育智能化中的应用潜力。

原文摘要 · Abstract (English)

Image captioning for Early Childhood Education (ECE) is essential for automated activity understanding and educational assessment. However, existing methods face two key challenges. First, the lack of large-scale, domain-specific datasets limits the model's ability to capture fine-grained semantic concepts unique to ECE scenarios, resulting in generic and imprecise descriptions. Second, conventional training paradigms exhibit limitations in enhancing professional object description capability, as supervised learning tends to favor high-frequency expressions, while reinforcement learning may suffer from unstable optimization on difficult samples. To address these limitations, we introduce ECAC, a large-scale benchmark for ECE daily activity image captioning, comprising 256,121 real-world images annotated with expert-level captions and fine-grained labels. ECAC is further equipped with a domain-oriented evaluation protocol, the Teaching Toy Recognition Score (TTS), to explicitly measure professional object naming accuracy. Furthermore, we propose RSRS (Reward-Conditional Switch of Reinforcement Learning and Supervised Fine-Tuning), a hybrid training framework that dynamically alternates between RL and supervised optimization. By rerouting hard samples with zero rewards to supervised fine-tuning, RSRS effectively mitigates advantage collapse and enables stable optimization for fine-grained recognition. Leveraging ECAC and RSRS, we develop KinderMM-Cap-3B, a domain-adapted multimodal large language model. Extensive experiments demonstrate that our model achieves a TTS of 51.06, substantially outperforming state-of-the-art baselines while maintaining superior caption quality, highlighting its potential for specialized educational applications.

图像描述幼儿教育多模态强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。