人类和机器对物体的混淆方向不同,揭示了各自不同的学习偏好。
Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision

- 通过混淆矩阵的不对称性分析视觉系统的内在偏见
- 人类混淆分布广但弱,机器则集中在少数类别上
- 即使准确率相同,两者泛化策略本质不同
人类和现代视觉模型在准确率上表现相当,但错误模式系统性不同:人觉得知更鸟更像鸟,而鸟不觉得更像知更鸟。我们通过12种自然图像扰动下的分类任务,量化了人类与深度神经网络在混淆矩阵中的方向性差异,并将其与信息-误差权衡的几何结构关联。结果显示,人类在多数类别对中表现出广泛但微弱的不对称性,而深度模型则呈现稀疏且强烈的定向坍缩至少数主导类别。鲁棒性训练虽降低了整体不对称程度,却无法恢复人类这种分散式结构。生成模拟进一步表明,二者在相同准确率下,其权衡几何方向相反,解释了为何相同的不对称性得分可能代表根本不同的泛化策略。这些结果确立了方向性混淆结构作为可解释、敏感的归纳偏见指标,是传统准确率评估无法捕捉的。
原文摘要 · Abstract (English)
To humans, a robin seems more like a bird than a bird seems like a robin, but does this asymmetry also hold for machine vision? Humans and modern vision models can match each other in accuracy while making systematically different kinds of errors, differing not in how often they fail, but in who gets mistaken for whom. We show these directional confusions reveal distinct inductive biases invisible to accuracy alone. Using matched human and deep neural network responses on a natural-image categorization task under 12 perturbation types, we quantify asymmetry in confusion matrices and link its organization to the geometry of the information--error trade-off - how efficiently, and how gracefully, a system generalizes under distortion. We find that humans exhibit broad but weak asymmetries across many class pairs, whereas deep vision models show sparser, stronger directional collapses into a few dominant categories. Robustness training reduces overall asymmetry magnitude but fails to recover this human-like distributed structure. Generative simulations further show that these two asymmetry organizations shift the trade-off geometry in opposite directions even at matched accuracy, explaining why the same scalar asymmetry score can reflect fundamentally different generalization strategies. Together, these results establish directional confusion structure as a sensitive, interpretable signature of inductive bias that accuracy-based evaluation cannot recover.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。