用可控生成修复视觉模型罕见情况错误，避免引入新问题。
SafeFix: Targeted Model Repair via Controlled Image Generation
- 基于失败归因定位罕见场景，用文本生成图像精准修复
- 重训练后罕见场景错误率下降47%，且无新错误产生
- 适合需要高可靠性视觉系统的开发人员使用
视觉识别的深度学习模型常因语义子群体代表性不足而出现系统性错误。现有调试框架虽能定位关键失败属性，但有效修复仍具挑战。当前方法依赖人工设计提示生成合成训练图像，易引发分布偏移与语义错误。为此，我们提出一个模型修复模块，基于可解释的失败归因流程，利用条件文本到图像模型生成语义准确、针对性强的失败案例图像。为保障生成样本的质量与相关性，进一步采用大视觉语言模型（LVLM）进行输出过滤，确保与原始数据分布对齐并保持语义一致性。通过在稀有案例增强的合成数据集上重新训练视觉模型，显著降低了罕见情况下的错误率。实验表明，该定向修复策略提升了模型鲁棒性，且未引入新错误。代码已开源：https://github.com/oxu2/SafeFix。
原文摘要 · Abstract (English)
Deep learning models for visual recognition often exhibit systematic errors due to underrepresented semantic subpopulations. Although existing debugging frameworks can pinpoint these failures by identifying key failure attributes, repairing the model effectively remains difficult. Current solutions often rely on manually designed prompts to generate synthetic training images -- an approach prone to distribution shift and semantic errors. To overcome these challenges, we introduce a model repair module that builds on an interpretable failure attribution pipeline. Our approach uses a conditional text-to-image model to generate semantically faithful and targeted images for failure cases. To preserve the quality and relevance of the generated samples, we further employ a large vision-language model (LVLM) to filter the outputs, enforcing alignment with the original data distribution and maintaining semantic consistency. By retraining vision models with this rare-case-augmented synthetic dataset, we significantly reduce errors associated with rare cases. Our experiments demonstrate that this targeted repair strategy improves model robustness without introducing new bugs. Code is available at https://github.com/oxu2/SafeFix
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。