用自反思智能体自动检测视觉模型依赖的特征
Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent
- 设计自反思智能体，迭代生成并验证特征依赖假设
- 在130个模型上验证，自反思使检测准确率显著提升
- 可发现CLIP和YOLOv8等主流模型的真实依赖关系
当视觉模型进行图像识别时，哪些视觉属性驱动其预测？检测对特定视觉特征的非预期依赖，对确保模型鲁棒性、防止过拟合及避免虚假相关至关重要。我们提出一种自动化框架，用于检测训练后视觉模型中的此类依赖。核心是自反思智能体，它系统性地生成并测试模型可能依赖的视觉属性假设。该过程为迭代式：智能体根据实验结果优化假设，并通过自我评估协议判断其发现是否准确解释模型行为。出现不一致时，智能体自我反思并启动新一轮实验。我们在一个包含130个模型的新基准上评估该方法，这些模型在18个类别中展现出多样化的视觉属性依赖。结果显示，随着自反思进行，智能体性能持续提升，显著优于无自反思基线。我们进一步证明，该智能体能识别出CLIP视觉编码器和YOLOv8目标检测器等前沿模型中的真实视觉属性依赖。
原文摘要 · Abstract (English)
When a vision model performs image recognition, which visual attributes drive its predictions? Detecting unintended reliance on specific visual features is critical for ensuring model robustness, preventing overfitting, and avoiding spurious correlations. We introduce an automated framework for detecting such dependencies in trained vision models. At the core of our method is a self-reflective agent that systematically generates and tests hypotheses about visual attributes that a model may rely on. This process is iterative: the agent refines its hypotheses based on experimental outcomes and uses a self-evaluation protocol to assess whether its findings accurately explain model behavior. When inconsistencies arise, the agent self-reflects over its findings and triggers a new cycle of experimentation. We evaluate our approach on a novel benchmark of 130 models designed to exhibit diverse visual attribute dependencies across 18 categories. Our results show that the agent's performance consistently improves with self-reflection, with a significant performance increase over non-reflective baselines. We further demonstrate that the agent identifies real-world visual attribute dependencies in state-of-the-art models, including CLIP's vision encoder and the YOLOv8 object detector.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。