arXiv:2605.24631cs.LGcs.AI2026-05

用世界模型引导扩散模型生成更符合真实语义的稀有样本。

Beyond Generative Priors: Minority Sampling with JEPA-Guided Diffusion

论文配图:Beyond Generative Priors: Minority Sampling with JEPA-Guided Diffusion
图 1 · 摘自论文原文
  • 用JEPA模型构建真实世界的先验,指导生成稀有实例。
  • 在多类生成任务中,生成样本的语义真实性和保真度显著提升。
  • 适合医疗诊断、异常检测等需精准稀有样本的场景。

少数类采样旨在生成数据流形上的低密度实例,在医学诊断、异常检测和创意AI中至关重要。现有方法将稀有性定义为模型学习到的生成先验下的罕见性,导致稀有性受限于特定模型,可能偏离真实语义。本文提出一种以世界为中心的少数类采样视角,基于真实世界先验而非生成器诱导的密度定义稀有性。为此,引入JEPA引导的扩散采样框架,利用联合嵌入预测架构(JEPA)——一种编码广泛语义信息的世界模型——来引导扩散轨迹,使其趋向于由JEPA隐式定义的低密度区域,从而生成与真实语义稀有性对齐的样本。为使计算可行,设计了合理的近似策略并提供理论误差界,显著降低引导计算开销。在无条件、类别条件及文生图生成任务上广泛实验表明,JEPA引导在保持高保真度的同时,更准确捕捉真实世界的稀有概念,优于生成器中心的基线方法。代码已开源。

原文摘要 · Abstract (English)

Minority sampling aims to generate low-density instances on a data manifold and is of central importance in applications such as medical diagnosis, anomaly detection, and creative AI. Existing approaches, however, define minority samples relative to generative priors learned from training data, confining rarity to model-specific notions that may poorly reflect real-world semantics. In this work, we propose a world-centric perspective on minority sampling, which defines rarity with respect to real-world priors rather than generator-induced densities. To this end, we introduce JEPA guidance, a diffusion sampling framework guided by a Joint-Embedding Predictive Architecture (JEPA) -- a class of world models that encode broad, semantically rich representations. JEPA guidance steers diffusion trajectories toward low-density regions under the implicit density induced by the JEPA, thereby aligning generated minorities with real-world semantic rarity. To make JEPA guidance computationally practical, we develop principled approximation strategies accompanied by theoretical error bounds, significantly reducing the overhead of guidance computation. Extensive experiments across unconditional, class-conditional, and text-to-image generation demonstrate that JEPA guidance consistently improves the fidelity and semantic validity of minority samples, outperforming generator-centric baselines in capturing real-world notions of rarity. Code is available at https://github.com/soobin-um/jepa-guidance.

扩散模型稀有样本世界模型生成艺术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。