揭示生成模型的结构偏差如何影响其生成能力
On the Anisotropy of Score-Based Generative Models
- 提出评分模型方向性偏差分析方法（SADs）
- SADs能预判模型泛化性能，与真实表现高度相关
- 适合研究生成模型机制与性能预测的学者
我们研究网络架构如何塑造现代评分生成模型的归纳偏置。为此，引入评分各向异性方向（SADs），这是一种依赖于架构的方向，可揭示不同网络对数据结构的偏好。分析表明，SADs形成与架构输出几何对齐的自适应基，为训练前预测评分模型的泛化能力提供了合理依据。通过合成数据和标准图像基准测试，我们证明SADs能可靠捕捉模型的细粒度行为，并与下游性能（以Wasserstein距离衡量）显著相关。本工作为解释和预测生成模型的方向性偏差提供了新视角。
原文摘要 · Abstract (English)
We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis suggests that SADs form adaptive bases aligned with the architecture's output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。