用掩码分离主体与背景,更准确评估图像生成中概念保真度和提示遵循度。
MaSC: A Masked Similarity Metric for Evaluating Concept-Driven Generation
- 通过外部提供的前景掩码,将图像分为主体和背景分别评估
- 在DreamBench++上概念保真度评分达Krippendorff alpha=0.471,接近GPT-4o
- 适用于需要精准衡量个性化生成质量的研究者和开发者
文本到图像扩散模型的单概念个性化评估需同时衡量概念保真度(身份忠实性)与提示遵循度。现有基于全局图像或图文嵌入(如CLIP-I、DINO、CLIP-T)的指标因关注整体图像,与人类感知相关性差。本文提出MaSC,利用外部提供前景掩码,将评估分解为:主体部分通过掩码最大余弦匹配计算概念保真度;背景部分通过仅含背景的图像嵌入与去主体提示嵌入对比衡量提示遵循度。使用冻结的SigLIP2 SO400M-NaFlex特征,实验显示:在DreamBench++上,概念保真度得分达到Krippendorff alpha=0.471,优于所有非LLM基线及GPT-4V,接近GPT-4o;在ORIDa真实照片身份保真基准上,AUC=0.992,几乎完美区分同主体与异主体对。提示遵循度也超越原版CLIP-T。结果表明,空间分解聚合是评估概念驱动生成的有效原则。
原文摘要 · Abstract (English)
Evaluating single-concept personalization in text-to-image diffusion requires measuring both concept preservation, which captures identity fidelity to a reference, and prompt following, which captures whether the generated scene matches the prompt. Existing metrics commonly compute these signals using global image or text-image embeddings, such as CLIP-I, DINO, and CLIP-T. We show that such metrics correlate poorly with human perception because they attend to the image as a whole instead of separating the concept subject from the background. We introduce MaSC, a masked similarity metric that uses externally provided foreground concept masks to decompose evaluation into subject-specific concept preservation and background-based prompt following. MaSC computes both scores from frozen SigLIP2 SO400M-NaFlex features: concept preservation is measured by masked max-cosine matching between foreground reference patches and generated-image patches, while prompt following is measured by comparing a background-only pooled image embedding to a subject-stripped prompt embedding. On DreamBench++ human ratings, MaSC achieves Krippendorff alpha = 0.471 for concept preservation, outperforming all tested non-LLM baselines and GPT-4V, and approaching GPT-4o. On ORIDa, a real-photo identity-preservation benchmark across physical environments, MaSC achieves AUC = 0.992, nearly perfectly distinguishing same-subject from cross-subject pairs. Its prompt-following score also outperforms the CLIP-T baseline shipped with DreamBench++. These results show that spatially decomposed aggregation is a strong design principle for evaluating concept-driven generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。