大模型越接近人眼感知，越能应对真实场景变化。
VFA: Vision Frequency Analysis of Foundation Models and Human
- 通过分析视觉模型与人类感知的匹配度，探索提升鲁棒性的方法。
- 模型规模、数据量和多模态信息显著提升与人类对齐程度。
- 适合关注模型泛化能力与人机对齐的研究者阅读。
机器学习模型在现实场景中常因分布偏移而表现不佳，而人类则具备出色的适应能力。若模型能更好对齐人类感知，可能实现更强的分布外泛化能力。本研究探讨大规模计算机视觉模型的多种特性如何影响其与人类能力的对齐程度及鲁棒性。实证分析表明，增大模型规模与数据量、融入丰富语义信息及多模态输入，可显著增强模型与人类感知的一致性，并提升整体鲁棒性。结果揭示出分布外准确率与人类对齐度之间存在强相关性。
原文摘要 · Abstract (English)
Machine learning models often struggle with distribution shifts in real-world scenarios, whereas humans exhibit robust adaptation. Models that better align with human perception may achieve higher out-of-distribution generalization. In this study, we investigate how various characteristics of large-scale computer vision models influence their alignment with human capabilities and robustness. Our findings indicate that increasing model and data size and incorporating rich semantic information and multiple modalities enhance models' alignment with human perception and their overall robustness. Our empirical analysis demonstrates a strong correlation between out-of-distribution accuracy and human alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。