arXiv:2603.25107cs.CV2026-03

动态调整多模态数据选择策略,提升标注效率与模型公平性

Label What Matters: Modality-Balanced and Difficulty-Aware Multimodal Active Learning

  • 用强化学习动态平衡各模态贡献,随训练过程自适应调整权重
  • 通过不确定性融合识别难样本,优先标注信息量大的数据
  • 在有限标注预算下兼顾准确率与模态公平性,适合资源受限场景

多模态学习依赖大规模标注数据,而主动学习可减少标注成本。现有方法通常假设模态重要性固定,忽略训练过程中模态价值和样本难度的动态变化。为此,本文提出RL-MBA框架,将样本选择建模为马尔可夫决策过程,通过策略自适应调节模态贡献、不确定性和多样性,奖励设计兼顾准确率提升与模态平衡。核心组件包括:(1) 自适应模态贡献平衡(AMCB),基于强化反馈动态调整模态权重;(2) 基于证据融合的难易度感知策略(EFDA),利用不确定性估计样本难度以优先选择高信息量样本。在Food101、KineticsSound和VGGSound数据集上的实验表明,RL-MBA持续优于强基线,在有限标注预算下显著提升分类准确率与模态公平性。

原文摘要 · Abstract (English)

Multimodal learning integrates complementary information from different modalities such as image, text, and audio to improve model performance, but its success relies on large-scale labeled data, which is costly to obtain. Active learning (AL) mitigates this challenge by selectively annotating informative samples. In multimodal settings, many approaches implicitly assume that modality importance is stable across rounds and keep selection rules fixed at the fusion stage, which leaves them insensitive to the dynamic nature of multimodal learning, where the relative value of modalities and the difficulty of instances shift as training proceeds. To address this issue, we propose RL-MBA, a reinforcement-learning framework for modality-balanced, difficulty-aware multimodal active learning. RL-MBA models sample selection as a Markov Decision Process, where the policy adapts to modality contributions, uncertainty, and diversity, and the reward encourages accuracy gains and balance. Two key components drive this adaptability: (1) Adaptive Modality Contribution Balancing (AMCB), which dynamically adjusts modality weights via reinforcement feedback, and (2) Evidential Fusion for DifficultyAware Policy Adjustment (EFDA), which estimates sample difficulty via uncertainty-based evidential fusion to prioritize informative samples. Experiments on Food101, KineticsSound, and VGGSound demonstrate that RL-MBA consistently outperforms strong baselines, improving both classification accuracy and modality fairness under limited labeling budgets.

主动学习多模态强化学习标注效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。