AI艺术风格识别与专家判断是否一致？
Does AI See like Art Historians? Interpreting How Vision Language Models Recognize Artistic Style
- 通过潜空间分解提取驱动风格预测的关键概念
- 73%概念被艺术史家认为语义清晰，90%相关性高
- 模型偶用无关概念仍准确，可能因形式特征理解
视觉语言模型在图像识别、问答等任务中表现优异，尤其在艺术领域展现出分析与生成能力。本文由计算机科学家与艺术史家合作，研究模型识别艺术风格的机制，并评估其与艺术史家判断标准的一致性。采用潜空间分解方法识别驱动风格预测的核心概念，结合量化评估、因果分析及艺术史家评审。结果显示，73%提取的概念被艺术史家评价为具有连贯且语义明确的视觉特征，90%用于预测特定作品风格的概念被认为相关。当模型使用无关概念仍成功预测时，艺术史家指出可能原因：模型或以更形式化方式理解概念，如明暗对比。
原文摘要 · Abstract (English)
VLMs have become increasingly proficient at a range of computer vision tasks, such as visual question answering and object detection. This includes increasingly strong capabilities in the domain of art, from analyzing artwork to generation of art. In an interdisciplinary collaboration between computer scientists and art historians, we characterize the mechanisms underlying VLMs' ability to predict artistic style and assess the extent to which they align with the criteria art historians use to reason about artistic style. We employ a latent-space decomposition approach to identify concepts that drive art style prediction and conduct quantitative evaluations, causal analysis and assessment by art historians. Our findings indicate that 73% of the extracted concepts are judged by art historians to exhibit a coherent and semantically meaningful visual feature and 90% of concepts used to predict style of a given artwork were judged relevant. In cases where an irrelevant concept was used to successfully predict style, art historians identified possible reasons for its success; for example, the model might "understand" a concept in more formal terms, such as dark/light contrasts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。