无需标注，让大模型自主发现视觉概念，提升可解释性与分类准确率。
S$^2$COPE: Self-Supervised Concept Discovery via Preference Learning
- 利用大模型在无标签图像中自动生成、验证并强化视觉属性，实现自监督概念发现。
- 在自然、医学和物理领域均有效提取新概念，下游分类准确率提升24个百分点。
- 适用于追求模型可解释性的研究者，尤其适合缺乏标注数据的场景。
当前表示学习面临根本性权衡：自监督方法可扩展至大规模数据集，但特征不透明；可解释模型则受限于密集的人工标注。我们提出自监督概念发现框架S²COPE，通过偏好学习机制，将视觉-语言大模型（VLLM）作为主动参与者，而非静态特征提取器。该框架直接从原始图像中自主假设、验证并强化候选视觉属性，完全无需人工标注即可发现新颖且结构化的概念。在自然、医疗及物理领域的大量实验表明，该方法能成功提取标准VLLM常无法生成的领域特定概念。通过将概念发现过程直接融入VLLM主干网络，基于自监督偏好目标，而非依赖静态生成与独立过滤，我们在未见数据上的下游分类任务中实现了最高达24个百分点的绝对准确率提升。本工作表明，可解释性可通过模型对偶然视觉结构的自主交互自发产生，无需任何人工监督。
原文摘要 · Abstract (English)
Current representation learning paradigms force a fundamental compromise: self-supervised methods scale to massive datasets but yield opaque features, whereas interpretable models remain bottlenecked by the need for dense human annotation. We introduce Self-Supervised Concept discOvery via Preference lEarning (\model), a label-free framework that resolves this dilemma. Instead of treating Vision-Large-Language Models (VLLMs) as static feature extractors, \model leverages them as active participants in a self-supervised preference optimization loop. By autonomously hypothesizing, validating, and reinforcing candidate visual attributes directly from raw imagery, our framework discovers novel, structured concepts without a single label. Extensive experiments across natural, medical, and physics domains demonstrate that \model successfully extracts domain-specific concepts where standard VLLMs often fail to generate. By amortizing concept discovery directly into the VLLM backbone through our self-supervised preference objective -- rather than relying on static generation and disjoint filtering -- we achieve up to a 24-point absolute improvement in downstream top-1 classification accuracy on unseen data. Our work suggest that interpretability can emerge through a model's autonomous interaction with incidental visual structures, without any human supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。