通过诊断驱动迭代训练,让大模型持续发现并弥补能力短板。
From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models
- 用多智能体自动标注生成多样真实数据,动态识别模型弱点。
- 在11个基准上实现稳定提升,模型能力随训练持续增强。
- 适合希望持续优化多模态模型的开发者与研究者使用。
随着大模型规模扩大和强化学习技术成熟,大视觉语言模型(LMMs)在复杂推理与决策任务中已取得显著进展。然而,当前训练仍依赖静态数据与固定流程,难以诊断能力盲区或提供动态、精准的强化。受测试驱动错误暴露与反馈修正优于重复练习的启发,我们提出诊断驱动渐进式演化(DPE),构建一个螺旋循环:诊断引导数据生成与强化,每轮迭代后重新诊断更新后的模型,推动下一轮针对性改进。DPE包含两个核心组件:首先,多个智能体利用网络搜索、图像编辑等工具,对海量无标签多模态数据进行标注与质量控制,生成多样化、真实样本;其次,DPE将失败归因于具体薄弱环节,动态调整数据组合,并引导智能体生成针对特定缺陷的数据以实现精准强化。在Qwen3-VL-8B-Instruct和Qwen2.5-VL-7B-Instruct上的实验表明,DPE在11个基准上均实现稳定且持续的性能提升,证明其在开放任务分布下具备可扩展的持续训练潜力。代码、模型与数据已公开于https://github.com/hongruijia/DPE。
原文摘要 · Abstract (English)
As Large Multimodal Models (LMMs) scale up and reinforcement learning (RL) methods mature, LMMs have made notable progress in complex reasoning and decision making. Yet training still relies on static data and fixed recipes, making it difficult to diagnose capability blind spots or provide dynamic, targeted reinforcement. Motivated by findings that test driven error exposure and feedback based correction outperform repetitive practice, we propose Diagnostic-driven Progressive Evolution (DPE), a spiral loop where diagnosis steers data generation and reinforcement, and each iteration re-diagnoses the updated model to drive the next round of targeted improvement. DPE has two key components. First, multiple agents annotate and quality control massive unlabeled multimodal data, using tools such as web search and image editing to produce diverse, realistic samples. Second, DPE attributes failures to specific weaknesses, dynamically adjusts the data mixture, and guides agents to generate weakness focused data for targeted reinforcement. Experiments on Qwen3-VL-8B-Instruct and Qwen2.5-VL-7B-Instruct show stable, continual gains across eleven benchmarks, indicating DPE as a scalable paradigm for continual LMM training under open task distributions. Our code, models, and data are publicly available at https://github.com/hongruijia/DPE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。