提出测试时动态融合新范式,理论保证更优泛化性能。
Test-Time Dynamic Image Fusion
- 从泛化视角构建动态融合框架,分解源图像成分
- 实验证明该方法降低泛化误差上界,提升融合鲁棒性
- 适合需要高可靠性的多源图像融合场景
图像融合的核心挑战在于捕捉多源图像的相关性并全面整合有效信息。现有方法大多无法实现动态融合,且缺乏理论保障,存在部署风险。本文从泛化角度出发,揭示图像融合的通用形式,推导出新的测试时动态图像融合范式,可证明降低泛化误差上界。具体地,将融合图像分解为对应源数据的多个分量,这些分量代表源数据中的有效信息,其差异反映单源数据在构建融合图像中的相对主导性(RD)。理论上证明,降低泛化误差的关键在于基于RD的融合权重与单源重建损失呈负相关。直观上,RD可动态突出各源图像的优势区域,并自然转化为融合权重,实现稳健融合。在多个基准上的大量实验及深入分析验证了该方法的有效性与优越性。代码已开源:https://github.com/Yinan-Xia/TTD。
原文摘要 · Abstract (English)
The inherent challenge of image fusion lies in capturing the correlation of multi-source images and comprehensively integrating effective information from different sources. Most existing techniques fail to perform dynamic image fusion while notably lacking theoretical guarantees, leading to potential deployment risks in this field. Is it possible to conduct dynamic image fusion with a clear theoretical justification? In this paper, we give our solution from a generalization perspective. We proceed to reveal the generalized form of image fusion and derive a new test-time dynamic image fusion paradigm. It provably reduces the upper bound of generalization error. Specifically, we decompose the fused image into multiple components corresponding to its source data. The decomposed components represent the effective information from the source data, thus the gap between them reflects the Relative Dominability (RD) of the uni-source data in constructing the fusion image. Theoretically, we prove that the key to reducing generalization error hinges on the negative correlation between the RD-based fusion weight and the uni-source reconstruction loss. Intuitively, RD dynamically highlights the dominant regions of each source and can be naturally converted to the corresponding fusion weight, achieving robust results. Extensive experiments and discussions with in-depth analysis on multiple benchmarks confirm our findings and superiority. Our code is available at https://github.com/Yinan-Xia/TTD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。