100次迭代编辑让图像严重失真,现有质量评估工具却察觉不了。
Banana100: Breaking NR-IQA Metrics by 100 Iterative Image Replications with Nano Banana Pro
- 通过100次迭代编辑生成2.8万张退化图像,模拟多轮编辑失效
- 21种无参考质量评估指标均无法有效识别严重退化图像
- 揭示多模态智能体在长期编辑中的脆弱性,适合关注AI安全的研究者
多模态智能体的多步迭代图像编辑能力已彻底改变数字内容创作。尽管最新图像编辑模型在单轮编辑中能准确遵循指令并生成高质量图像,我们发现其在多轮编辑中存在关键缺陷:图像质量随反复编辑持续退化。微小瑕疵不断累积,迅速导致明显噪声和基础指令失效。为系统研究此类失败,我们构建了香蕉100(Banana100)数据集,包含通过100次迭代编辑生成的28,000张退化图像,涵盖多样纹理与内容。令人警觉的是,图像质量评估器未能检测到退化。在21种主流无参考图像质量评估(NR-IQA)指标中,没有一种能一致地为重度退化图像赋予比原始图像更低的分数。生成器与评估器的双重失效可能威胁未来模型训练的稳定性及部署式智能体的安全性,若由多轮编辑生成的低质合成数据逃过质量过滤。我们开源全部代码与数据,以促进更鲁棒模型的发展,缓解多模态智能体的脆弱性。
原文摘要 · Abstract (English)
The multi-step, iterative image editing capabilities of multi-modal agentic systems have transformed digital content creation. Although latest image editing models faithfully follow instructions and generate high-quality images in single-turn edits, we identify a critical weakness in multi-turn editing, which is the iterative degradation of image quality. As images are repeatedly edited, minor artifacts accumulate, rapidly leading to a severe accumulation of visible noise and a failure to follow simple editing instructions. To systematically study these failures, we introduce Banana100, a comprehensive dataset of 28,000 degraded images generated through 100 iterative editing steps, including diverse textures and image content. Alarmingly, image quality evaluators fail to detect the degradation. Among 21 popular no-reference image quality assessment (NR-IQA) metrics, none of them consistently assign lower scores to heavily degraded images than to clean ones. The dual failures of generators and evaluators may threaten the stability of future model training and the safety of deployed agentic systems, if the low-quality synthetic data generated by multi-turn edits escape quality filters. We release the full code and data to facilitate the development of more robust models, helping to mitigate the fragility of multi-modal agentic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。