arXiv:2608.05608cs.LGcs.AI2026-08

提出细粒度证据门控机制,解决多模态分类中缺失数据的可靠性问题。

GAUGE: Granularity-Adaptive Counterfactual Gating of Evidence for Incomplete Multimodal Classification

论文配图:GAUGE: Granularity-Adaptive Counterfactual Gating of Evidence for Incomplete Multimodal Classification
图 1 · 摘自论文原文
  • 通过反事实评分对细粒度证据单元进行动态加权
  • 在六个基准上优于现有方法,提升不完整输入下的分类精度
  • 无需修改主干网络,适合部署于实际多模态系统

多模态分类通常假设所有模态均可用,但现实输入常存在缺失。现有方法在粗粒度模态层面进行补全和融合,无法保留同一模态中可靠的组件并抑制误导性成分,影响预测可靠性。为此,本文提出GAUGE,一种轻量级反事实门控框架,用于不完整多模态分类。GAUGE首先使用冻结的插补器恢复缺失模态,并将观测与恢复的输入统一编码为细粒度证据单元。不直接干预每个单元,而是通过预测感知的泰勒证据分数,在一次前向-反向传播中评估替换每个单元为参考表示的反事实效应。这些分数映射为连续门控,转换为加性注意力偏置,实现单元级证据调制,且无需修改骨干架构。六项基准实验表明,GAUGE在多种不完整输入设置下超越强基线。此外,泰勒余项理论分析刻画了一阶近似误差相对于精确反事实效应的关系,确立了GAUGE在模态不完整场景下细粒度证据控制的原理性与可扩展性。

原文摘要 · Abstract (English)

Multimodal classification typically assumes all modalities are available, yet real-world inputs are often incomplete. Imputation and dynamic fusion can mitigate such incompleteness, but existing methods operate at a coarse modality level and thus cannot retain reliable components while suppressing misleading ones within the same recovered modality, compromising prediction reliability. To address this issue, we propose GAUGE, a lightweight counterfactual gating framework for incomplete multimodal classification. GAUGE first imputes missing modalities with a frozen imputer and encodes observed and recovered inputs uniformly as fine-grained evidence units. Rather than intervening on each unit explicitly, GAUGE scores the counterfactual effect of replacing every unit with a reference representation through prediction-aware Taylor evidence scores, all obtained in a single forward-backward pass. These scores are mapped to continuous gates, which are converted into additive attention-logit biases for unit-wise evidence modulation without altering the backbone architecture. Experiments across six benchmarks demonstrate that GAUGE outperforms strong baselines across diverse incomplete-input settings. Furthermore, a Taylor remainder theoretical analysis characterizes the error of the first-order approximation relative to the exact counterfactual effect, establishing GAUGE as a principled and scalable framework for fine-grained evidence control under modality incompleteness.

多模态学习反事实推理缺失数据证据门控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。