用真实背景+3D玻璃路径追踪，生成逼真单图反射去除数据集
SIRR-LMM: Single-image Reflection Removal via Large Multimodal Model
- 合成数据通过3D玻璃路径追踪与真实背景结合，物理精度高
- 使用联合描述和LoRA微调，提升大模型在去反射任务上的性能
- 适合做图像修复、计算机视觉中需要去反射的应用
玻璃表面会产生复杂的反射与透射光交互，导致单图反射去除（SIRR）极具挑战。现有数据集或合成数据物理真实感不足，或真实采集规模有限。本文提出一种合成数据生成框架，将3D玻璃模型在真实背景图像上进行路径追踪，生成具有多样玻璃属性、相机设置和后期处理效果的物理准确反射场景。为利用大模态模型（LMM）能力，我们将图像层拼接为单一复合输入，进行联合描述，并采用任务特定的LoRA微调而非全参数训练。该方法在反射去除与分离性能上优于当前最优方法。
原文摘要 · Abstract (English)
Glass surfaces create complex interactions of reflected and transmitted light, making single-image reflection removal (SIRR) challenging. Existing datasets suffer from limited physical realism in synthetic data or insufficient scale in real captures. We introduce a synthetic dataset generation framework that path-traces 3D glass models over real background imagery to create physically accurate reflection scenarios with varied glass properties, camera settings, and post-processing effects. To leverage the capabilities of Large Multimodal Model (LMM), we concatenate the image layers into a single composite input, apply joint captioning, and fine-tune the model using task-specific LoRA rather than full-parameter training. This enables our approach to achieve improved reflection removal and separation performance compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。