通过智能分配视觉令牌,大幅减少3D医学图像处理的计算负担。
CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding

- 按跨切片证据动态分配令牌,非均匀压缩冗余信息。
- 移除80%令牌后仍保持98.1%性能,优于所有基线方法。
- 适合需要高效3D医学图像理解的医疗AI研究者使用。
基于切片的多模态大模型利用成熟的2D编码器,将3D医学体积数据表示为2D切片序列。然而,这种切片式处理生成数千个视觉令牌,导致大型语言模型负担过重,且相邻切片间存在大量重叠信息。我们在两个3D医学视觉问答基准上进行扩展分析,发现性能提升呈边际递减:成本持续上升而准确率趋于饱和;提升平面分辨率比增加切片数量更有效。因此,应更智能地分配令牌预算而非盲目扩大。现有压缩方法多针对2D图像或视频中的空间/时间冗余,不适用于深度轴上的近似重复内容。我们提出CARVE,一种无需训练的框架,在推理前压缩视觉令牌,并将令牌削减视为受预算约束的2.5D分配问题。CARVE将深度轴划分为一致窗口,依据归一化跨切片证据非均匀分配令牌。在共享预算下,于代表性切片建立空间锚点,从全体积检索局部变化的证据,并将剩余合格令牌合并至各窗口内的邻近锚点。在Hulu-Med-7B上移除约80%的视觉令牌后,CARVE在所有AMOS-MM报告生成指标上领先所有压缩基线,相比最强基线保留6.2分的完整令牌性能,且在三个VQA基准上维持98.1%的完整性能。
原文摘要 · Abstract (English)
Slice-based MLLMs leverage mature 2D encoders by representing 3D volumes as sequences of 2D slices. However, this slice-wise formulation produces thousands of visual tokens that burden the LLM backbone, many of which capture overlapping visual evidence across adjacent slices. To understand how effectively a growing visual token budget improves performance, we perform scaling analyses on two 3D medical VQA benchmarks and find diminishing returns: cost keeps rising while accuracy saturates, and improving in-plane resolution is more effective than adding slices at comparable budgets. The budget should therefore be allocated more selectively rather than simply enlarged, yet most token compression methods are designed for 2D images or videos, where redundancy arises from spatial layout or temporal motion rather than from near-duplicate content along the depth axis. We present CARVE, a training-free framework that compresses visual tokens prior to LLM inference and casts token reduction as budget-constrained 2.5D allocation. CARVE partitions the depth axis into coherent windows and allocates tokens non-uniformly according to normalized cross-slice evidence. Under a shared budget, CARVE builds spatial anchors on representative slices and retrieves locally varying evidence from the full volume, then merges remaining eligible tokens into nearby anchors within each window. Removing roughly 80% of the visual tokens on Hulu-Med-7B, CARVE leads all compression baselines on every AMOS-MM report-generation metric, with 6.2 points higher retention of full-token quality than the strongest baseline, and preserves 98.1% of full-token performance across three VQA benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。