动态图像处理暴露隐私，攻击者可从运行时间与缓存差异推断输入内容。
Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Models
- 利用动态分块导致的执行时间波动，通过系统指标识别图像几何特征
- 通过缓存竞争分析区分视觉密集与稀疏内容，突破相同形状的语义模糊
- 适用于关注边缘AI隐私安全的研究者与开发者
本地执行的视觉语言模型（VLMs）承诺数据隐私保护。然而，我们发现向动态高分辨率预处理（如AnyRes）的架构演进引入了固有的算法侧信道。与静态模型不同，动态预处理根据图像长宽比将图像分解为可变数量的图像块，生成依赖工作负载的输入。本文提出针对本地VLM的双层攻击框架：第一层中，普通用户可通过标准操作系统指标捕捉显著的执行时间差异，可靠地指纹识别输入的几何特征；第二层中，通过分析最后一级缓存（LLC）争用情况，可解决相同几何结构下的语义模糊问题，从而区分视觉密集内容（如医学X光片）与视觉稀疏内容（如文本文档）。通过对最新模型如LLaVA-NeXT和Qwen2-VL的评估，我们证明结合这些信号可可靠推断出敏感上下文。最后，我们分析了缓解该漏洞的安全工程权衡，揭示恒定工作量填充带来的显著性能开销，并提出了面向安全边缘AI部署的实用设计建议。
原文摘要 · Abstract (English)
On-device Vision-Language Models (VLMs) promise data privacy via local execution. However, we show that the architectural shift toward Dynamic High-Resolution preprocessing (e.g., AnyRes) introduces an inherent algorithmic side-channel. Unlike static models, dynamic preprocessing decomposes images into a variable number of patches based on their aspect ratio, creating workload-dependent inputs. We demonstrate a dual-layer attack framework against local VLMs. In Tier 1, an unprivileged attacker can exploit significant execution-time variations using standard unprivileged OS metrics to reliably fingerprint the input's geometry. In Tier 2, by profiling Last-Level Cache (LLC) contention, the attacker can resolve semantic ambiguity within identical geometries, distinguishing between visually dense (e.g., medical X-rays) and sparse (e.g., text documents) content. By evaluating state-of-the-art models such as LLaVA-NeXT and Qwen2-VL, we show that combining these signals enables reliable inference of privacy-sensitive contexts. Finally, we analyze the security engineering trade-offs of mitigating this vulnerability, reveal substantial performance overhead with constant-work padding, and propose practical design recommendations for secure Edge AI deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。