用生成式语义抗体让视觉语言模型更敢说‘不知道’，提升开放世界可信度。
Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness
- 通过大模型生成近似异常的文本描述作为语义抗体，主动界定已知类边界。
- 在ImageNet-1K和四个外部数据集上均达到新最优性能，显著提升对未知样本的识别能力。
- 适合关注模型鲁棒性、开放世界推理与可信AI的研究者和应用开发者。
大型视觉语言模型通过将视觉特征与广泛语义概念对齐，在零样本识别中取得了前所未有的成功。然而，这种语义抽象带来了开放世界部署中的关键漏洞：‘语义自负’——模型因缺乏显式的负向知识，会以高置信度将未知异常强行归入已有类别。为解决这一‘开放世界可信度悖论’，我们提出**Immuno-VLM**，一种受生物免疫机制启发的框架，将**免疫学负选择原理**应用于高维潜在空间。不同于依赖被动密度估计或低效像素空间异常生成的传统开放集识别方法，Immuno-VLM 利用大语言模型的生成推理能力，主动幻化出‘语义抗体’——即近分布外异常（如外观相似物、上下文异常）的文本描述，有效约束已知类的决策空间。在ImageNet-1K及四个挑战性外部数据集上的大量实验表明，Immuno-VLM 建立了新的最先进水平。
原文摘要 · Abstract (English)
Large Vision-Language Models have achieved unprecedented success in zero-shot recognition by aligning visual features with broad semantic concepts. However, this semantic abstraction creates a critical vulnerability in open-world deployment: the ``Hubris of Semantics'', where models force-fit unknown anomalies into known categories with high confidence due to the lack of explicit negative knowledge. To address this \textit{Open-World Trustworthiness Paradox}, we propose \textbf{Immuno-VLM}, a bio-inspired framework that adapts the biological principle of \textbf{Immunological Negative Selection} to high-dimensional latent spaces. Departing from traditional Open-Set Recognition methods that rely on passive density estimation or inefficient pixel-space outlier generation, Immuno-VLM leverages the generative reasoning of Large Language Models to actively hallucinate ``Semantic Antibodies'', textual descriptions of near-distribution outliers (e.g., look-alikes, contextual anomalies) that effectively bound the decision space of known classes.Extensive experiments on ImageNet-1K and four challenging OOD benchmarks reveal that Immuno-VLM establishes a new state-of-the-art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。