让视觉语言模型更快更省内存，不降精度。
CIVIC: End-to-End Sequence Compactness for Efficient Vision-Language Models
- 设计连续紧凑的视觉推理路径，全程保持高效序列结构。
- KV缓存内存减至1/3，端到端推理延迟显著降低。
- 适合追求高效部署的视觉语言模型研究与应用者。
视觉语言模型（VLMs）因高分辨率视觉标记导致严重的内存和延迟瓶颈。现有标记压缩方法虽理论上节省计算量，但后期剪枝引入结构开销，无法带来相应实际加速。同时，强制连续紧凑路径可能造成几何错位和细粒度定位损失。为此，本文提出CIVIC——一种路径一致的紧凑视觉推理框架。通过在视觉编码器、投影层、LLM预填充及键值缓存中无缝保持紧凑序列表示，CIVIC避免了非连续内存访问与局部解合并开销。在Qwen3-VL架构上评估表明，CIVIC将键值缓存内存缩减至基线的约三分之一，并显著降低端到端推理延迟。得益于文本对齐的KL蒸馏与自适应空间保留下限，该方法在多模态推理与视觉定位基准测试中未牺牲准确性。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) face severe memory and latency bottlenecks due to high-resolution visual tokens. While current token reduction methods theoretically save FLOPs, post-hoc pruning introduces structural overhead, failing to yield proportional wall-clock acceleration. However, enforcing a contiguous compact pathway risks geometric disorientation and loss of fine-grained localization. To overcome these barriers, this paper introduces CIVIC, a path-consistent compact visual inference framework. By maintaining compact sequence representations seamlessly across the vision encoder, projection layer, LLM prefill, and KV-cache, CIVIC avoids non-contiguous memory access and localized unmerging overheads. Evaluated on the Qwen3-VL architecture, CIVIC successfully translates sequence reductions into genuine physical hardware efficiency, shrinking KV-cache memory to approximately one-third of the baseline and reducing end-to-end inference latency. Enabled by text-aligned KL distillation and an adaptive spatial retention floor, CIVIC achieves these efficiency milestones without degrading accuracy across rigorous multimodal reasoning and visual grounding benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。