arXiv:2607.29637cs.CVcs.SE2026-07

通过智能压缩代码图像,大幅减少模型输入量且不损失理解能力。

CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding

论文配图:CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
图 1 · 摘自论文原文
  • 用紧凑布局和结构标记替代空白区域,减少无效视觉令牌
  • 动态调整压缩策略,实现令牌效率与可读性的平衡
  • 根据任务指令自动剔除无关代码视觉信息,适合高效代码理解场景

将源代码渲染为图像可有效降低多模态大模型的输入成本。调整图像分辨率可在视觉令牌数量与内容保真度间权衡,但仅靠分辨率缩放忽略了两个低效因素:换行和缩进造成的空白区域,以及与当前任务无关的代码部分。此外,最佳压缩设置随输入、任务和模型变化,固定比例策略受限。我们提出CodeShrink,一个包含三部分的自适应视觉压缩框架:无空白渲染采用紧凑布局与显式结构标记,消除布局带来的冗余令牌;自适应压缩配置通过轻量级强化学习代理,为每个输入预测最优压缩设置;主导令牌选择在推理时联合分析指令与代码图像,剔除任务无关的视觉信息。我们在代码问答、代码克隆检测和代码补全任务上评估,CodeShrink 最多减少71.2%视觉令牌,同时达到或超过未压缩纯文本输入表现,并在所有任务中持续优于基于文本和视觉压缩的基线方法。结果表明,结合布局精简、自适应配置与指令感知裁剪,能显著提升多模态代码理解效率。代码已开源。

原文摘要 · Abstract (English)

Rendering source code as images offers a promising way to reduce the input costs of Multimodal Large Language Models (MLLMs). Adjusting image resolution can trade visual token cost against content fidelity. However, resolution scaling alone overlooks two sources of inefficiency: blank regions created by line breaks and indentation, and code regions irrelevant to the current instruction. Moreover, the best compression setting varies across inputs, tasks, and models, limiting fixed-ratio strategies. We propose CodeShrink, an adaptive visual compression framework with three components. Blank-Free Rendering replaces whitespace-dependent layouts with compact layouts and explicit structural markers, removing layout-induced tokens. Adaptive Compression Configuration uses a lightweight agent trained with reinforcement learning to predict a per-input setting that balances token efficiency and readability. Dominant Token Selection jointly analyzes the instruction and code image to prune task-irrelevant visual tokens during inference. We evaluate CodeShrink on code question answering, clone detection, and code completion. CodeShrink reduces visual token use by up to 71.2\% while matching or exceeding uncompressed text-only inputs, and consistently outperforms text-based and visual compression baselines across all three tasks. These results show that combining layout compaction, adaptive configuration, and instruction-aware pruning can make multimodal code understanding more efficient. Our code is available at https://github.com/vinsontang1/CodeShrink.

代码理解视觉压缩多模态高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。