arXiv:2602.11596cs.AI2026-02

让多模态模型学会按需使用信息,训练更快更稳。

MAPLE: Modality-Aware Post-training and Learning Ecosystem

  • 按任务需求区分模态重要性,分组优化降低梯度噪声。
  • 准确率差距缩小30.24%,收敛速度提升3.18倍。
  • 适合需要真实场景鲁棒性的多模态系统部署。

多模态语言模型现整合文本、音频和视频以实现统一推理。然而现有强化学习后训练流程将所有输入信号视为同等重要,忽略任务实际所需的模态组合。这种模态盲训练导致策略梯度方差增大,收敛变慢,并在现实分布偏移(如信号缺失、新增或重加权)下性能下降。我们提出MAPLE,一个完整的模态感知后训练与学习生态:(1) MAPLE-bench,首个显式标注各任务最小所需信号组合的基准;(2) MAPO,一种按模态需求分层批处理的策略优化框架,减少异质组优势带来的梯度方差;(3) 自适应加权与课程调度机制,平衡并优先处理更难的信号组合。在损失聚合、裁剪、采样与课程设计上的系统分析确立了MAPO的最优训练策略。自适应加权与课程聚焦学习进一步提升各类信号组合下的表现。MAPLE将单/多模态准确率差距缩小30.24%,收敛速度提升3.18倍,并在真实信号受限条件下保持全部模态组合的稳定性。MAPLE构成可直接部署的多模态强化学习后训练完整方案。

原文摘要 · Abstract (English)

Multimodal language models now integrate text, audio, and video for unified reasoning. Yet existing RL post-training pipelines treat all input signals as equally relevant, ignoring which modalities each task actually requires. This modality-blind training inflates policy-gradient variance, slows convergence, and degrades robustness to real-world distribution shifts where signals may be missing, added, or reweighted. We introduce MAPLE, a complete modality-aware post-training and learning ecosystem comprising: (1) MAPLE-bench, the first benchmark explicitly annotating minimal signal combinations required per task; (2) MAPO, a modality-aware policy optimization framework that stratifies batches by modality requirement to reduce gradient variance from heterogeneous group advantages; (3) Adaptive weighting and curriculum scheduling that balances and prioritizes harder signal combinations. Systematic analysis across loss aggregation, clipping, sampling, and curriculum design establishes MAPO's optimal training strategy. Adaptive weighting and curriculum focused learning further boost performance across signal combinations. MAPLE narrows uni/multi-modal accuracy gaps by 30.24%, converges 3.18x faster, and maintains stability across all modality combinations under realistic reduced signal access. MAPLE constitutes a complete recipe for deployment-ready multimodal RL post-training.

多模态强化学习后训练模态感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。