比较猫与人视觉表征，发现自监督ViT最接近
Purrturbed but Stable: Human-Cat Invariant Representations Across CNNs, ViTs and Self-Supervised ViTs
- 用多种模型对比猫与人视觉特征相似性
- DINO ViT-B/16对齐度最高，达0.814（RBF）
- 自监督学习让模型更贴近跨物种视觉机制
猫与人类眼结构差异显著，猫的竖瞳与伏击捕猎相关。本文构建统一基准，使用冻结编码器量化卷积网络、监督ViT、窗口化Transformer及自监督ViT（DINO）在野外条件下猫与人视觉表征的跨物种对齐度，采用层间中心核对齐（线性与RBF）和表征相似性分析，并补充分布与稳定性测试。结果显示，DINO ViT-B/16表现最优，平均CKA-RBF≈0.814，平均CKA-linear≈0.745，平均RSA≈0.698，且在早期块达到峰值，表明令牌级自监督催生了能弥合物种统计差异的早期特征。监督ViT在CKA上表现尚可，但几何对应弱于DINO（如ViT-B/16在第8块时RSA≈0.53，ViT-L/16在第14块时≈0.47），揭示相似性与表征几何间的深度依赖差异。CNN仍为强基线但低于普通ViT，窗口化Transformer性能更差，说明架构归纳偏置影响跨物种对齐。结果表明，自监督结合ViT归纳偏置可生成更贴近猫与人视觉系统的表征几何，为跨物种视觉计算的收敛点提供可检验的神经科学假说。代码与数据集已开源。
原文摘要 · Abstract (English)
Cats and humans differ in ocular anatomy. Most notably, Felis Catus (domestic cats) have vertically elongated pupils linked to ambush predation; yet, how such specializations manifest in downstream visual representations remains incompletely understood. We present a unified, frozen-encoder benchmark that quantifies feline-human cross-species representational alignment in the wild, across convolutional networks, supervised Vision Transformers, windowed transformers, and self-supervised ViTs (DINO), using layer-wise Centered Kernel Alignment (linear and RBF) and Representational Similarity Analysis, with additional distributional and stability tests reported in the paper. Across models, DINO ViT-B/16 attains the most substantial alignment (mean CKA-RBF $\approx0.814$, mean CKA-linear $\approx0.745$, mean RSA $\approx0.698$), peaking at early blocks, indicating that token-level self-supervision induces early-stage features that bridge species-specific statistics. Supervised ViTs are competitive on CKA yet show weaker geometric correspondence than DINO (e.g., ViT-B/16 RSA $\approx0.53$ at block8; ViT-L/16 $\approx0.47$ at block14), revealing depth-dependent divergences between similarity and representational geometry. CNNs remain strong baselines but below plain ViTs on alignment, and windowed transformers underperform plain ViTs, implicating architectural inductive biases in cross-species alignment. Results indicate that self-supervision coupled with ViT inductive biases yields representational geometries that more closely align feline and human visual systems than widely used CNNs and windowed Transformers, providing testable neuroscientific hypotheses about where and how cross-species visual computations converge. We release our code and dataset for reference and reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。