视觉微扰可骗过价格约束，让多模态智能体误购高价商品
Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations
- 用语义解耦损失生成难以察觉的图像干扰，绕过价格判断
- 在真实电商场景中使智能体误判率超80%，跨模型攻击成功率35%-41%
- 揭示现有模型在价格敏感任务中的脆弱性，适合安全评估与防御研究者
多模态大模型的快速普及使移动代理能够执行高风险金融交易，但其对抗鲁棒性仍待探索。我们发现视觉主导幻觉（VDH）现象：在基于截图的价格约束场景中，难以察觉的视觉线索可覆盖文本价格信息，导致代理做出非理性决策。为此提出PriceBlind，一种隐蔽的白盒对抗攻击框架，用于受控截图评估。该方法通过语义解耦损失，在保持像素级保真度的前提下，将图像嵌入对齐至低成本、价值相关锚点，利用CLIP编码器的模态差距实现攻击。在E-ShopBench上，PriceBlind在白盒测试中达到约80%的攻击成功率；在简化单轮坐标选择协议下，Ensemble-DI-FGSM对GPT-4o、Gemini-1.5-Pro和Claude-3.5-Sonnet的迁移攻击成功率约为35%-41%。我们还发现，使用鲁棒编码器和验证后执行（Verify-then-Act）防御可显著降低攻击成功率，但会带来一定的干净准确率损失。
原文摘要 · Abstract (English)
The rapid proliferation of Multimodal Large Language Models (MLLMs) has enabled mobile agents to execute high-stakes financial transactions, but their adversarial robustness remains underexplored. We identify Visual Dominance Hallucination (VDH), where imperceptible visual cues can override textual price evidence in screenshot-based, price-constrained settings and lead agents to irrational decisions. We propose PriceBlind, a stealthy white-box adversarial attack framework for controlled screenshot-based evaluation. PriceBlind exploits the modality gap in CLIP-based encoders via a Semantic-Decoupling Loss that aligns the image embedding with low-cost, value-associated anchors while preserving pixel-level fidelity. On E-ShopBench, PriceBlind achieves around 80% ASR in white-box evaluation; under a simplified single-turn coordinate-selection protocol, Ensemble-DI-FGSM transfers with roughly 35-41% ASR across GPT-4o, Gemini-1.5-Pro, and Claude-3.5-Sonnet. We also show that robust encoders and Verify-then-Act defenses reduce ASR substantially, though with some clean-accuracy trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。