用扩散模型探测数据潜在分层结构，揭示噪声下的关联变化规律。
Probing the Latent Hierarchical Structure of Data via Diffusion Models
- 通过扩散模型的加噪-去噪过程探测数据隐含分层结构
- 发现数据变化以关联块形式出现，块长度在相变点处发散
- 在文本与图像数据中验证了该现象，适用于结构分析任务
高维数据必须具有高度结构化才具备可学习性。尽管数据的组合性与分层特性常被用来解释可学习性，但量化测量此类性质的研究仍很少。同时，获取支撑这些结构的潜在变量也面临挑战。本文表明，基于扩散模型的前向-反向实验（即对数据加噪后去噪生成新样本）是探测数据潜在结构的有力工具。我们在简单分层模型中预测，在此过程中数据变化以相关块的形式发生，且块的尺度在已知相变发生的噪声水平处趋于发散。令人惊讶的是，我们使用最先进的扩散模型在文本和图像数据集中验证了这一预测。结果展示了潜在变量变化如何体现在数据中，并建立了利用扩散模型测量真实数据中此类效应的方法。
原文摘要 · Abstract (English)
High-dimensional data must be highly structured to be learnable. Although the compositional and hierarchical nature of data is often put forward to explain learnability, quantitative measurements establishing these properties are scarce. Likewise, accessing the latent variables underlying such a data structure remains a challenge. In this work, we show that forward-backward experiments in diffusion-based models, where data is noised and then denoised to generate new samples, are a promising tool to probe the latent structure of data. We predict in simple hierarchical models that, in this process, changes in data occur by correlated chunks, with a length scale that diverges at a noise level where a phase transition is known to take place. Remarkably, we confirm this prediction in both text and image datasets using state-of-the-art diffusion models. Our results show how latent variable changes manifest in the data and establish how to measure these effects in real data using diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。