测试视觉语言模型在无纹理情况下的几何理解能力，发现其严重依赖颜色纹理。
BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs
- 用纯轮廓图测试模型对几何形状的识别能力
- 26个主流模型在无颜色时性能普遍骤降，平均下降超50%
- 适合关注模型真实理解能力的研究者与开发者
尽管视觉语言模型在多模态任务中展现出出色的零样本识别能力，但其是否真正理解几何结构仍存疑。现有评估方法无法区分语义推理与纹理映射，且标注不精准导致环境线索泄露。为此，我们提出BareBones，一个专为测试纯几何理解设计的零样本基准。我们收集了六个数据集（ImageNet-S、DIS5K、ThinObject5K、PASCAL VOC、CUB-200）及新构建的WTP-Bench中的像素级轮廓图，建立无噪声几何分类体系。WTP-Bench是一个精细的视觉谜题，仅依赖边界轮廓区分类别。对26个先进模型（如GPT-4.1、Gemini、Claude Sonnet 4.5、LLaVA）的评估显示，在去除RGB信息后性能普遍崩溃，我们称之为‘纹理偏见悬崖’。该结果揭示了模型普遍存在的结构盲区，为真正的几何建模提供了严格度量标准。
原文摘要 · Abstract (English)
While Vision-Language Models (VLMs) demonstrate remarkable zero-shot recognition capabilities across a diverse spectrum of multimodal tasks, it yet remains an open question whether these architectures genuinely comprehend geometric structure or merely exploit RGB textures and contextual priors as statistical shortcuts. Existing evaluations fail to isolate this mechanism, conflating semantic reasoning with texture mapping and relying on imprecise annotations that inadvertently leak environmental cues. To address this gap, we introduce $\textbf{BareBones}$, a zero-shot benchmark designed to stress-test pure geometric shape comprehension. We curate pixel-level silhouettes of geometrically distinct classes across six datasets: five established segmentation sources (ImageNet-S, DIS5K, ThinObject5K, PASCAL VOC, CUB-200) and our novel flagship collection, WTP-Bench, establishing a noise-free geometric taxonomy. WTP-Bench is an extreme, fine-grained visual puzzle that forces models to identify inter-class geometric concepts from boundary contours alone. Our evaluation of 26 state-of-the-art proprietary and open-weight VLMs (eg. GPT-4.1, Gemini, Claude Sonnet 4.5, LLaVA) reveals a consistent, severe performance collapse under RGB deprivation, a phenomenon we term the $\textit{Texture Bias Cliff}$. By documenting universal structural blindspots, BareBones establishes a rigorous yardstick for genuine geometric grounding. Project Page: https://eternal-f1ame.github.io/WTP-Bench/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。