arXiv:2605.06708cs.CVcs.AI2026-05

用测度传输理论分析图文压缩的信息损失,提升长文本处理效率。

Visual Text Compression as Measure Transport

论文配图:Visual Text Compression as Measure Transport
图 1 · 摘自论文原文
  • 将图文压缩建模为测度传输问题,分解出精度与覆盖两类成本
  • 无需下游标签即可判断是否使用视觉路径,准确率达70.8%
  • 提出聚焦机制,在高成本区域提高分辨率以优化效果

视觉文本压缩(VTC)通过将文本渲染为图像并用视觉语言模型重编码,实现长上下文高效处理,通常可减少3到20倍的解码器标记数。然而,压缩比并不稳定预测下游性能:某些任务中视觉路径表现优于或匹配文本路径,另一些则严重退化,且压缩比本身无法预判结果。关键缺失的是对任务相关信息损失的合理度量。本文以测度传输语言重新建模VTC,将文本与视觉标记视为经验概率测度,证明ViT块编码器诱导一个前推映射,其运输成本可分解为块内聚合的精度成本和块间碎片化的覆盖成本。这两项均可通过无下游标签的探测器估计。该框架带来两个实际应用:一是无需标签的路由准则,决定何时使用视觉路径;二是基于运输成本的聚焦机制,在高成本区域以更高分辨率重编码。在Qwen3-4B模型上测试24个NLP数据集,该无标签规则在17/24个数据集上达到每数据集最优表现(70.8%),平均任务得分提升+3.3%,同时平均标记数减少-10.3%。

原文摘要 · Abstract (English)

Visual text compression (VTC) promises efficient long-context processing by rendering text into an image and re-encoding it with a vision-language model, often producing $3$--$20\times$ fewer decoder tokens than subword tokenization. Yet token savings do not translate predictably into downstream utility: on some tasks the visual path matches or exceeds the text path, on others it collapses, and the compression ratio itself does not predict which regime will occur. The missing quantity is therefore not another summary of efficiency, but a principled measure of task-relevant information loss induced by visual encoding. We address this problem by formulating VTC in the language of measure transport. Treating text and visual tokens as empirical probability measures, we show that the ViT patch encoder induces a push-forward map whose transport cost decomposes into a precision cost from within-patch aggregation and a coverage cost from cross-patch fragmentation. Both terms are estimable from downstream-label-free probes. This formulation yields two operational consequences: a downstream-label-free routing criterion that selects whether to use the visual path for a given input or benchmark instance, and a transport-informed foveation mechanism that re-encodes high-cost regions at higher resolution. Across $24$ NLP datasets at Qwen3-4B, our label-free rule matches the per-dataset oracle on $17/24$ datasets ($70.8\%$), and improves the average task score by $+3.3\%$ with $-10.3\%$ average tokens relative to a pure-LLM.

视觉文本压缩测度传输高效推理多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。