用参考图修复缝合，让拼接图像无痕过渡。
Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpainting
- 以参考图为指导，通过修复式融合实现图像拼接
- 在零样本测试中仍保持强泛化能力，拼接更自然
- 无需标注数据，利用文本生成模型自监督训练
当前图像拼接方法在色差不均和大视差等挑战场景下常产生明显接缝。为此，我们提出参考驱动修复拼接器（RDIStitcher），将图像融合与校正重构为基于参考的修复模型，采用更大修改区域和更强修改强度。此外，引入自监督训练方法，通过微调文本到图像扩散模型实现无需标注数据的训练。针对拼接图像质量评估难题，提出基于多模态大语言模型（MLLMs）的评价指标，提供新视角。大量实验表明，相比现有最先进方法，本方法显著提升拼接图像的内容连贯性与无缝衔接效果，尤其在零样本实验中展现强大泛化能力。代码已开源。
原文摘要 · Abstract (English)
Current image stitching methods often produce noticeable seams in challenging scenarios such as uneven hue and large parallax. To tackle this problem, we propose the Reference-Driven Inpainting Stitcher (RDIStitcher), which reformulates the image fusion and rectangling as a reference-based inpainting model, incorporating a larger modification fusion area and stronger modification intensity than previous methods. Furthermore, we introduce a self-supervised model training method, which enables the implementation of RDIStitcher without requiring labeled data by fine-tuning a Text-to-Image (T2I) diffusion model. Recognizing difficulties in assessing the quality of stitched images, we present the Multimodal Large Language Models (MLLMs)-based metrics, offering a new perspective on evaluating stitched image quality. Compared to the state-of-the-art (SOTA) method, extensive experiments demonstrate that our method significantly enhances content coherence and seamless transitions in the stitched images. Especially in the zero-shot experiments, our method exhibits strong generalization capabilities. Code: https://github.com/yayoyo66/RDIStitcher
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。