发现扩散模型中被忽略的稀有属性,无需预先定义类别。
RAIGen: Rare Attribute Identification in Text-to-Image Generative Models
- 用自编码器和新指标找模型里隐藏的稀有特征神经元。
- 在Stable Diffusion和SDXL中识别出非主流文化/风格属性。
- 适合做生成模型审计或个性化内容生成的研究者使用。
文本到图像扩散模型虽生成质量高,却继承并放大训练数据偏见,导致语义属性覆盖失衡。现有方法分两类:封闭集方法针对预定义公平类别(如性别、种族)缓解偏见,假设少数群体属性已知;开放集方法将任务视为偏见识别,突出主导性属性。两者均忽视一个互补任务:发现数据分布中被低估的稀有或少数特征(社会、文化或风格层面),但仍在模型表征中存在。本文提出RAIGen,据我们所知首个无需标签的扩散模型稀有属性发现框架,无需预设少数类别。RAIGen利用马特约什卡稀疏自编码器与一种结合神经元激活频率与语义独特性的新度量,识别可解释神经元,其最高激活图像揭示了未被覆盖的属性。实验表明,RAIGen可在Stable Diffusion中发现超出固定公平类别的属性,扩展至SDXL等更大模型,支持跨架构系统性审计,并实现生成阶段对稀有属性的定向增强。项目页面见 https://vssilpa.github.io/RAIGen_webpage/。
原文摘要 · Abstract (English)
Text-to-image diffusion models achieve impressive generation quality but inherit and amplify training-data biases, skewing coverage of semantic attributes. Prior work addresses this in two ways. Closed-set approaches mitigate biases in predefined fairness categories (e.g., gender, race), assuming socially salient minority attributes are known a priori. Open-set approaches frame the task as bias identification, highlighting majority attributes that dominate outputs. Both overlook a complementary task: uncovering rare or minority features underrepresented in the data distribution (social, cultural, or stylistic) yet still encoded in model representations. We introduce RAIGen, the first framework, to our knowledge, for label-free rare-attribute discovery in diffusion models, requiring no predefined minority categories. RAIGen leverages Matryoshka Sparse Autoencoders and a novel minority metric combining neuron activation frequency with semantic distinctiveness to identify interpretable neurons whose top-activating images reveal underrepresented attributes. Experiments show RAIGen discovers attributes beyond fixed fairness categories in Stable Diffusion, scales to larger models such as SDXL, supports systematic auditing across architectures, and enables targeted amplification of rare attributes during generation. The project page is available at https://vssilpa.github.io/RAIGen_webpage/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。