一个模型同时生成真实感图像和内部属性，支持多任务编辑。
PRISM: A Unified Framework for Photorealistic Reconstruction and Intrinsic Scene Modeling
- 用微调方法让单个模型同步输出图像与内在属性层。
- 在分解与生成任务上表现优异，且保留原始文本生成能力。
- 适合需要统一处理图像生成与编辑的开发者使用。
我们提出 PRISM，一个统一框架，可在单一基础模型中实现多种图像生成与编辑任务。基于预训练的文本到图像扩散模型，PRISM 提出一种高效的微调策略，可同步生成 RGB 图像及内在属性图（称为 X 层）。与以往需单独建模或分阶段推断的方法不同，PRISM 通过联合生成所有内在层，确保多模态间的一致性。该框架支持文本到 RGBX 生成、RGB 到 X 分解以及 X 到 RGBX 条件生成等多种任务。此外，通过条件输入特定内在层与文本提示，可实现全局与局部图像编辑。大量实验表明，PRISM 在内在图像分解与条件生成任务上均表现优秀，同时保持了基模型的文本到图像生成能力。
原文摘要 · Abstract (English)
We present PRISM, a unified framework that enables multiple image generation and editing tasks in a single foundational model. Starting from a pre-trained text-to-image diffusion model, PRISM proposes an effective fine-tuning strategy to produce RGB images along with intrinsic maps (referred to as X layers) simultaneously. Unlike previous approaches, which infer intrinsic properties individually or require separate models for decomposition and conditional generation, PRISM maintains consistency across modalities by generating all intrinsic layers jointly. It supports diverse tasks, including text-to-RGBX generation, RGB-to-X decomposition, and X-to-RGBX conditional generation. Additionally, PRISM enables both global and local image editing through conditioning on selected intrinsic layers and text prompts. Extensive experiments demonstrate the competitive performance of PRISM both for intrinsic image decomposition and conditional image generation while preserving the base model's text-to-image generation capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。