arXiv:2501.09052eess.IVcs.LG2025-01IJCV被引 4

用因果孪生网络实现图像去模糊的持续在线自适应,提升泛化能力。

Continual Test-Time Adaptation for Single Image Defocus Deblurring via Causal Siamese Networks

  • 构建因果孪生网络,利用视觉语言模型提取语义先验,增强适应性。
  • 在连续变化的测试域上,显著提升现有去模糊方法的性能。
  • 适合需要实时适应新镜头模糊特性的图像恢复场景。

单图散焦去模糊(SIDD)旨在从模糊图像中恢复全清晰图像。由于模糊图像的分布偏移,现有方法在分布外推理时性能下降。本文发现性能退化源于镜头特异的点扩散函数差异,并通过实验证明该假设。为此,提出一种持续测试时自适应(CTTA)框架,仅需无标签目标数据即可在线更新模型。传统基于熵最小化的CTTA方法难以捕捉像素级回归任务的依赖信息,因此设计基于孪生网络的因果自适应方法(CauSiam)。该方法利用大规模预训练视觉-语言模型获取通用语义先验,并将其融入孪生结构,确保模糊输入与重建图像间的因果可识别性。大量实验表明,CauSiam能有效提升现有SIDD方法在持续变化域中的泛化能力。

原文摘要 · Abstract (English)

Single image defocus deblurring (SIDD) aims to restore an all-in-focus image from a defocused one. Distribution shifts in defocused images generally lead to performance degradation of existing methods during out-of-distribution inferences. In this work, we gauge the intrinsic reason behind the performance degradation, which is identified as the heterogeneity of lens-specific point spread functions. Empirical evidence supports this finding, motivating us to employ a continual test-time adaptation (CTTA) paradigm for SIDD. However, traditional CTTA methods, which primarily rely on entropy minimization, cannot sufficiently explore task-dependent information for pixel-level regression tasks like SIDD. To address this issue, we propose a novel Siamese networks-based continual test-time adaptation framework, which adapts source models to continuously changing target domains only requiring unlabeled target data in an online manner. To further mitigate semantically erroneous textures introduced by source SIDD models under severe degradation, we revisit the learning paradigm through a structural causal model and propose Causal Siamese networks (CauSiam). Our method leverages large-scale pre-trained vision-language models to derive discriminative universal semantic priors and integrates these priors into Siamese networks, ensuring causal identifiability between blurry inputs and restored images. Extensive experiments demonstrate that CauSiam effectively improves the generalization performance of existing SIDD methods in continuously changing domains.

图像去模糊持续学习因果推断视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。