arXiv:2608.07141cs.CV2026-08

用游戏色彩空间测试模型与人类对颜色的感知一致性,发现模型在抽象领域严重偏差。

Human-AI Perceptual Alignment by Playing Hues and Cues

论文配图:Human-AI Perceptual Alignment by Playing Hues and Cues
图 1 · 摘自论文原文
  • 用桌游Hues and Cues的离散色彩空间构建评估框架
  • 162个模型中多数在抽象和流行文化领域偏离人类基准
  • 小而精的训练数据比海量无序数据更有效提升对齐

评估对比视觉-语言模型(CVLMs)与人类的感知一致性,常受限于忽略细粒度语义和文化差异的传统基准。本文提出一种新评估框架,利用桌游Hues and Cues的480个色彩单元映射至CIE xy色度图,基于涵盖七个语义类别的100词词汇集计算实证感知距离。为准确定位模型表现,通过325名人类观察者使用自研数字界面收集密集色彩关联数据,采用留一法(LOO)交叉验证建立预期误差的实证下限——人类一致性基线。我们在多个架构家族和预训练数据集上评估了162个模型的语义色彩锚定能力。结果表明,尽管模型能复现人类认知偏见(如食物与植物的理想化记忆色),但在抽象、主观及流行文化领域系统性偏离人类基准。我们识别出两类严重错配:语义误分类和不确定性坍缩至默认蓝色坐标。此外,高度精选的预训练数据显著优于大规模未标注语料库,可有效缓解此类错配。本研究揭示,当前CVLM虽具广泛分类能力,仍未能捕捉人类色彩记忆的细微局部共识,凸显游戏化任务在暴露模型深层偏见方面的价值。数据与代码已公开,可供测试其他度量方法。

原文摘要 · Abstract (English)

Evaluating the perceptual alignment between Contrastive Vision-Language Models (CVLMs) and humans is typically constrained by traditional benchmarks that overlook fine-grained semantic and cultural nuances. In this work, we propose a novel evaluation framework that leverages the gamified, discrete color space of the board game Hues and Cues. By mapping the board's 480 color cells to the CIE xy chromaticity diagram, we calculate empirical perceptual distances across a carefully curated 100-word vocabulary spanning seven semantic categories. To properly contextualize model performance, we establish an empirical lower bound of expected error-the Human Consistency baseline-calculated via Leave-One-Out (LOO) cross-validation on a dense dataset of color associations collected from 325 human observers through a custom digital interface. We evaluate 162 models across multiple architectural families and pre-training datasets to assess their semantic color grounding. Our results demonstrate that while CVLMs successfully replicate human cognitive biases, such as idealized memory colors for concrete physical referents (e.g., food and plants), they systematically diverge from the human baseline in abstract, subjective, and pop-culture domains. We identify two distinct failure modes in severely misaligned concepts: semantic misclassification and a systematic uncertainty collapse into a default blue coordinate. Furthermore, we reveal that highly curated pre-training datasets are significantly more effective than massive, uncurated corpora in mitigating these severe misalignments. Ultimately, this work highlights that despite their broad categorization capabilities, current CVLMs still fail to capture the nuanced, localized consensus of human color memory, emphasizing the value of gamified tasks in exposing underlying model biases. The data and code are publicly available to test other metrics.

感知对齐视觉语言模型色彩感知游戏化评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。