揭示视觉模型训练中相似性演变规律,理解错误如何随时间改进
Inspecting Training Dynamics of Similarity Development in Supervised Vision Networks
- 提出DSI框架,动态追踪模型训练中相似性的变化
- 发现两类网络均经历相似性三阶段演化:激增、优化、稳定
- 首次观察到错误随训练时间自我修正的现象,具可解释性
为实现可信且以人为本的人工智能,模型评估需超越准确率,关注错误可预测性与语义对齐。相似性是这些方面核心,影响模型认为哪些类别相关或易混淆。相似性呈现多种形式,包括可作为人类感知代理的语义相似性。尽管相似性感知常被人为引入计算机视觉,但其在监督训练中的自然涌现却少受关注。现有研究多局限于静态定性分析,缺乏系统性训练时视角。因此,本文分析了相似性感知如何在监督视觉网络中演进,并与模型错误模式及语义对齐。为此,我们提出深度相似性检查器(Deep Similarity Inspector, DSI)——一个统一互补视角的系统性、训练时框架。利用DSI,我们分析了卷积与基于Transformer的网络,发现两者均通过三个阶段发展出丰富的相似性结构:初始相似性激增、精炼与稳定,同时展现出显著差异。我们还识别出‘错误精炼’现象:网络随时间逐步改进自身错误。
原文摘要 · Abstract (English)
For trustworthy and human-aware artificial intelligence, models should be evaluated beyond accuracy, among others through error predictability and semantic alignment. Similarity is central to these aspects, as it influences which classes a model considers related and confusable. Similarity manifests in multiple forms, including semantic similarity, which can serve as a proxy for human similarity perception. While similarity perception is often imposed in computer vision, little attention has been paid to its natural emergence during supervised training. Existing studies are largely limited to static and qualitative analyzes and lack a systematic, training-time perspective. Therefore, we analyze how similarity perception evolves and aligns with model error patterns and semantics in supervised vision networks. As an enabler, we introduce Deep Similarity Inspector (DSI) - a systematic, training-time framework that unifies complementary views on similarity within a single methodology. Using DSI, we analyzed Convolutional and Transformer-based Networks and showed that both architectures develop rich similarity structures through three phases - initial similarity surge, refinement, stabilization - while exhibiting clear differences. We also identified the mistake refinement phenomenon, in which networks improve mistakes with time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。