探索生成模型如何提升视觉表征,推动应用落地。
Representation Learning in Diffusion and Flow-based Model: An Application Aspect
- 提出三层次框架,系统梳理生成与表征的双向关系。
- 涵盖分类、密集预测等任务,覆盖标注稀缺场景。
- 适合关注生成模型应用潜力的研究者参考。
扩散模型和流模型已成为生成建模的主流范式,主要得益于其通过大规模训练学习丰富多层视觉表征的能力。这催生了生成模型与表征学习之间的双向互动:提升表征学习可增强生成质量,而学习到的表征可用于更广泛的感知任务。本文系统探讨这一互动关系,聚焦应用层面。提出一个三层次渐进式框架,从三个角度组织现有工作:利用表征学习提升生成能力;利用生成模型提取表征用于感知任务;最终迈向通用统一的应用。系统分类了涵盖图像分类、密集视觉预测、实例级感知及标注稀缺场景在内的多种下游任务的代表性方法。通过构建统一分类体系并识别关键挑战,旨在厘清当前研究内在逻辑,提出未来探索方向。期望本工作能为希望利用生成模型表征能力拓展应用的研究者提供重要参考。
原文摘要 · Abstract (English)
Diffusion models and flow-based models have recently become the dominant paradigms in generative modeling, largely due to their ability to learn rich, multi-level visual representations through large-scale training. This creates a bidirectional relationship between generative models and representation learning: improving representation learning enhances generation quality, while the learned representations can be leveraged for broader understanding tasks. This survey systematically explores this interplay with a focus on applications. We propose a three-tier progressive framework that organizes existing works from three perspectives: using representation learning to improve generative capabilities, exploiting generative models to extract representations for perception tasks, and ultimately moving toward general-purpose unified applications. We systematically categorize representative methods across a wide range of downstream tasks, including image classification, dense visual prediction, instance-level perception, and annotation-scarce scenarios. By providing a unified taxonomy and identifying key challenges, this survey aims to clarify the underlying logic of current research and suggest promising directions for future exploration. We hope this work can serve as a valuable reference for researchers interested in harnessing the representation power of generative models for applications beyond generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。