让生成图像准确呈现透明物体的折射效果，突破传统模型局限。
Refracting Reality: Generating Images with Realistic Transparent Objects
- 用斯涅尔定律在生成过程中同步内外像素，实现物理正确折射
- 通过全景图辅助恢复折射/反射中不可见表面的外观
- 显著提升透明物体真实感，适合需要高精度视觉生成的场景
生成式图像模型能生成形状、纹理、布局和光照都逼真的图像，但在合成透明物体方面表现较差，因其涉及折射、反射、吸收和散射等光学现象。其中折射尤为困难，因为折射后的像素光线会与图像其他部分的表面相交，对颜色形成约束。观察表明，生成模型尚未充分掌握光学规律，无法准确渲染折射物体。本文研究如何根据文本提示生成具有精确折射效果的图像。我们通过在生成轨迹的每一步使用斯涅尔定律对物体边界内与外的像素进行形变与融合，实现像素级同步。对于图像中未直接观测到但可通过折射或反射看到的表面，我们通过将图像与一个以物体为中心的全景图进行相同操作来恢复其外观。实验表明，该方法生成的图像在光学上更合理，严格遵守物理约束。
原文摘要 · Abstract (English)
Generative image models can produce convincingly real images, with plausible shapes, textures, layouts and lighting. However, one domain in which they perform notably poorly is in the synthesis of transparent objects, which exhibit refraction, reflection, absorption and scattering. Refraction is a particular challenge, because refracted pixel rays often intersect with surfaces observed in other parts of the image, providing a constraint on the color. It is clear from inspection that generative models have not distilled the laws of optics sufficiently well to accurately render refractive objects. In this work, we consider the problem of generating images with accurate refraction, given a text prompt. We synchronize the pixels within the object's boundary with those outside by warping and merging the pixels using Snell's Law of Refraction, at each step of the generation trajectory. For those surfaces that are not directly observed in the image, but are visible via refraction or reflection, we recover their appearance by synchronizing the image with a second generated image -- a panorama centered at the object -- using the same warping and merging procedure. We demonstrate that our approach generates much more optically-plausible images that respect the physical constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。