揭示逆问题中参数估计的理论边界,挑战盲目去模糊提升估计效果的常见做法。
On Inverse Problems, Parameter Estimation, and Domain Generalization
- 区分连续与离散参数估计,分析直接从观测值估计与先恢复再估计的差异
- 证明在非可逆退化下,数据处理反而可能降低参数估计性能,符合信息论不等式
- 提出双义定理,揭露当前域泛化方法在离散参数任务中的根本缺陷
信号恢复与逆问题是绝大多数真实世界数据科学应用的核心。过去几十年中,随着机器学习方法的兴起,测量反演已成为几乎所有物理应用中的关键步骤,通常在下游参数估计任务前执行。本文提出一个通用的理论框架,用于分析逆问题设置下的参数估计。我们区分了连续与离散参数估计,分别对应回归与分类问题。针对可逆与不可逆退化过程,研究直接从观测值进行参数估计与先对观测值进行反演再估计的对比。理论结果与经典信息论数据处理不等式一致,并在一定程度上质疑了‘基于现代生成模型的高质量感知恢复必能提升后续参数估计’这一普遍误解。尤为重要的是,通过将域偏移问题重新表述为离散参数估计的关联形式,我们揭示了当前主流域泛化策略的重大脆弱性,称之为‘双义定理’。这些理论发现通过图像去模糊和医学成像中散斑抑制的域偏移案例得到实验验证。我们希望本工作能为从业者提供更深入的洞察,以指导未来高效且明智的系统规划,尤其在安全敏感场景中至关重要。
原文摘要 · Abstract (English)
Signal restoration and inverse problems are key elements in most real-world data science applications. In the past decades, with the emergence of machine learning methods, inversion of measurements has become a popular step in almost all physical applications, normally executed prior to downstream tasks that often involve parameter estimation. In this work, we propose a general framework for theoretical analysis of parameter estimation in inverse problem settings. We distinguish between continuous and discrete parameter estimation, corresponding with regression and classification problems, respectively. We investigate this setting for invertible and non-invertible degradation processes, with parameter estimation that is executed directly from the observed measurements, comparing with parameter estimation after data-processing performing an inversion of the observations. Our theoretical findings align with the well-known information-theoretic data processing inequality, and to a certain degree question the common misconception that data-processing for inversion, based on modern generative models that may often produce outstanding perceptual quality, will necessarily improve the following parameter estimation objective. Importantly, by re-formulating the domain-shift problem in direct relation with discrete parameter estimation, we expose a significant vulnerability in current popular practical attempts to enforce domain generalization, which we dubbed the Double Meaning Theorem. These theoretical findings are experimentally illustrated for domain shift examples in image deblurring and speckle suppression in medical imaging. It is our hope that this paper will provide practitioners with deeper insights that may be leveraged in the future for the development of more efficient and informed strategic system planning, critical in safety-sensitive applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。