构建鸟类身份保持生成数据集,提升非刚性物种的图像生成准确性。
Not All Birds Look The Same: Identity-Preserving Generation For Birds
- 按物种、年龄、性别分组训练,作为身份代理提升生成一致性。
- 在4759对专家标注的鸟图上,现有模型身份保持能力显著不足。
- 适用于需要高精度细节的生物图像生成,如生态研究与虚拟观察。
可控图像生成技术不断进步,零样本身份保持模型(如Insert Anything和OminiControl)已支持虚拟试穿等应用,无需额外微调。然而,这些方法在非刚性或细粒度类别中仍存在局限,尤其缺乏高质量视频或多视角数据,导致难以评估与优化。鸟类是理想测试领域:种类多样、识别依赖细粒度特征、姿态多变。本文构建了包含4,759对专家标注图像的NABirds Look-Alikes(NABLA)数据集,结合iNaturalist上的1,073对多图观测数据及少量视频,形成鸟类身份保持生成的基准。实验表明,现有先进基线在该数据集上无法有效保持身份,而基于物种、年龄、性别分组训练(作为身份代理)可显著提升对已见与未见物种的生成表现。
原文摘要 · Abstract (English)
Since the advent of controllable image generation, increasingly rich modes of control have enabled greater customization and accessibility for everyday users. Zero-shot, identity-preserving models such as Insert Anything and OminiControl now support applications like virtual try-on without requiring additional fine-tuning. While these models may be fitting for humans and rigid everyday objects, they still have limitations for non-rigid or fine-grained categories. These domains often lack accessible, high-quality data -- especially videos or multi-view observations of the same subject -- making them difficult both to evaluate and to improve upon. Yet, such domains are essential for moving beyond content creation toward applications that demand accuracy and fine detail. Birds are an excellent domain for this task: they exhibit high diversity, require fine-grained cues for identification, and come in a wide variety of poses. We introduce the NABirds Look-Alikes (NABLA) dataset, consisting of 4,759 expert-curated image pairs. Together with 1,073 pairs collected from multi-image observations on iNaturalist and a small set of videos, this forms a benchmark for evaluating identity-preserving generation of birds. We show that state-of-the-art baselines fail to maintain identity on this dataset, and we demonstrate that training on images grouped by species, age, and sex -- used as a proxy for identity -- substantially improves performance on both seen and unseen species.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。