arXiv:2509.25178cs.CVcs.AI2025-09被引 2

用自动优化隐含提示生成诱导幻觉的图像,测试多模态模型漏洞。

GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs

  • 在图像嵌入空间优化生成误导性线索,无需人工干预。
  • 使模型幻觉成功率超28%,远高于此前方法的约1%。
  • 可发现跨模型通用漏洞,且可用于训练缓解幻觉问题。

多模态大模型中的物体幻觉是其持续存在的缺陷,导致模型感知图像中并不存在的物体。当前研究依赖静态基准和固定视觉场景,难以发现模型特异性或未预见的幻觉漏洞。我们提出GHOST(通过优化隐蔽标记生成幻觉),一种全自动方法,通过主动生成诱导幻觉的图像来压力测试多模态大模型。GHOST在图像嵌入空间中优化,以误导模型但保持目标物体不存在,并引导扩散模型根据嵌入生成自然图像。生成图像视觉自然、接近原始输入,却包含细微误导线索,引发模型幻觉。我们在多个模型上评估,包括推理型模型GLM-4.1V-Thinking,幻觉成功率达28%以上,远超此前数据驱动方法的约1%。定量指标与人工评估均证实生成图像质量高且无目标物体。此外,针对Qwen2.5-VL优化的图像在GPT-4o上引发66.5%的幻觉率,表明存在可迁移漏洞。最后,我们证明在这些图像上微调可有效缓解幻觉,使GHOST兼具诊断与矫正功能。

原文摘要 · Abstract (English)

Object hallucination in Multimodal Large Language Models (MLLMs) is a persistent failure mode that causes the model to perceive objects absent in the image. This weakness of MLLMs is currently studied using static benchmarks with fixed visual scenarios, which preempts the possibility of uncovering model-specific or unanticipated hallucination vulnerabilities. We introduce GHOST (Generating Hallucinations via Optimizing Stealth Tokens), a method designed to stress-test MLLMs by actively generating images that induce hallucination. GHOST is fully automatic and requires no human supervision or prior knowledge. It operates by optimizing in the image embedding space to mislead the model while keeping the target object absent, and then guiding a diffusion model conditioned on the embedding to generate natural-looking images. The resulting images remain visually natural and close to the original input, yet introduce subtle misleading cues that cause the model to hallucinate. We evaluate our method across a range of models, including reasoning models like GLM-4.1V-Thinking, and achieve a hallucination success rate exceeding 28%, compared to around 1% in prior data-driven discovery methods. We confirm that the generated images are both high-quality and object-free through quantitative metrics and human evaluation. Also, GHOST uncovers transferable vulnerabilities: images optimized for Qwen2.5-VL induce hallucinations in GPT-4o at a 66.5% rate. Finally, we show that fine-tuning on our images mitigates hallucination, positioning GHOST as both a diagnostic and corrective tool for building more reliable multimodal systems.

多模态模型幻觉检测扩散模型漏洞挖掘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。