提升扩散模型逆问题求解的可靠性，防止生成与测量不符的虚假细节。
Hallucination-Aware Diffusion Sampling for Inverse Problems via Robust Prior Updates
- 通过稳定先验更新步骤，防止早期生成不合理的图像内容。
- 在FFHQ数据集上，PSNR和LPIPS指标均优于基线方法，人眼评估偏好率达91.9%。
- 特别适合对生成内容真实性要求高的图像修复、去模糊任务。
基于扩散模型的逆问题求解器虽能生成逼真结果，但真实感不等于符合测量数据。本文将此类失败归因于测量条件下的幻觉：视觉上合理却与观测数据矛盾的内容。分析表明，基于贝叶斯规则的扩散求解器可分为先验更新与测量条件化两步，幻觉主要源于先验更新阶段的不稳定提议。为此提出鲁棒先验更新（RPU）模块，探测先验更新的局部稳定性，重新锚定位移至当前迭代点，保持测量更新不变。在DPS框架中实现RPU，于FFHQ与ImageNet上评估自动指标与人工忠实度判断。在FFHQ上，箱式补全、高斯模糊去除及运动模糊去除任务中，RPU在PSNR与LPIPS上均优于DPS；人眼评估中，盲评非平局偏好率达91.9%，真值辅助非平局偏好为91.1%；ImageNet高斯模糊研究虽平局较多，但非平局情况下仍倾向RPU。结果支持核心观点：强化先验更新的鲁棒性可显著提升扩散逆问题求解的实例忠实度，尤其在先验约束弱的情况下。
原文摘要 · Abstract (English)
Diffusion-based inverse problem solvers can produce realistic reconstructions, but realism alone does not ensure that the recovered details are supported by the measurement. We study this failure as measurement-conditioned hallucination: visually meaningful content that is either implausible or inconsistent with the measured instance. Our analysis separates Bayes-rule-based diffusion inverse solvers into a prior update and a measurement-conditioning step, showing that hallucinated content can enter through the prior-side proposal before the measurement correction is applied. Motivated by this view, we propose Robust Prior Update (RPU), a solver-level module that probes the local stability of the diffusion prior update, re-anchors the resulting displacement at the current iterate, and leaves the measurement update unchanged. We instantiate RPU in DPS and evaluate it on FFHQ and ImageNet inverse problems using automatic metrics and human faithfulness studies. On FFHQ, RPU improves PSNR and LPIPS over DPS across box inpainting, Gaussian deblurring, and motion deblurring. In human judgments, RPU receives 91.9% of blind non-tie majority preferences and 91.1% of ground-truth-assisted non-tie preferences on FFHQ box inpainting, while the ImageNet Gaussian reader study is tie-heavy but favors RPU among non-tie cases. These results support a targeted claim: robustifying the prior update can improve instance faithfulness in diffusion inverse solvers, especially when the prior shapes weakly constrained content.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。