arXiv:2607.27952cs.CVcs.AI2026-07

用最后一个查询词指导剪枝,让边缘设备高效减少传给云端的视觉数据。

LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference

论文配图:LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference
图 1 · 摘自论文原文
  • 通过查询词末尾注意力生成轻量剪枝信号,无需访问云端模型。
  • 仅保留12.5%视觉令牌,仍保持95.4%完整精度,显著降低云端计算量。
  • 适合资源受限的边缘设备,尤其适用于多模态大模型协同推理场景。

多模态基础模型正将边缘-云视觉智能从任务特定特征流水线转变为基于标记的接口,其中边缘设备将视觉输入编码为标记供通用云端多模态大模型(MLLM)处理。然而,密集的视觉标记序列增加了云端推理成本。现有剪枝方法主要针对集中式推理:视觉驱动方法可在云端执行前操作,但通常对查询不敏感;查询引导方法常依赖目标MLLM内部状态,无法在传输前判断标记相关性。紧凑的引导模型是替代方案,但现有设计可能需要昂贵的注意力聚合或辅助生成。我们提出LAST,一种无需训练的边缘-云协同多模态大模型推理中查询相关的视觉标记剪枝框架。LAST使用边缘侧小型多模态模型作为引导代理,从最后一个查询标记对视觉标记的注意力中提取轻量重要性信号。在因果注意力机制下,最后一个查询标记可关注完整视觉序列及全部查询上下文,实现无需云端模型访问、自回归生成或多个查询位置的昂贵聚合的查询感知剪枝。LAST在固定标记预算下保留多样化的查询相关视觉标记。我们在11个多模态基准上,在多种标记预算下评估LAST,对比不同引导策略的剪枝方法。实验表明,LAST始终表现最佳,在仅保留12.5%视觉标记的同时,保持95.4%的完整标记准确率,边缘侧选择开销低,云端计算显著减少。

原文摘要 · Abstract (English)

Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge devices encode visual inputs into tokens for a general-purpose cloud MLLM. However, dense visual-token sequences increase cloud-side inference costs. Existing pruning methods mainly target centralized inference: vision-driven methods can operate before cloud execution but are typically query-agnostic, whereas query-guided methods often rely on internal states of the target MLLM and cannot determine token relevance before transmission. Compact guidance models offer an alternative, but existing designs may require costly attention aggregation or auxiliary generation. We propose LAST, a training-free framework for query-dependent visual token pruning in edge-cloud collaborative MLLM inference. LAST uses a compact edge-side VLM as a guidance proxy and derives a lightweight importance signal from the last query token's attention to visual tokens. Under causal attention, the last query token can attend to the full visual sequence and the entire query context, enabling query-aware pruning without cloud-model access, autoregressive generation, or costly aggregation over multiple query positions. LAST then retains a diverse set of query-relevant visual tokens under a fixed token budget. We evaluate LAST on 11 multimodal benchmarks under multiple token budgets against pruning methods with different guidance strategies. Experiments show that LAST consistently achieves the strongest performance, preserving 95.4% of the full-token accuracy while retaining only 12.5% of the visual tokens, with low edge-side selection overhead and reduced cloud-side computation.

边缘计算视觉剪枝多模态高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。