先分配资源再排序，让音视频各取所需，压缩效率更高
Allocation Before Ranking: Decoupled Token Compression for OmniLLMs
- 先定音视频预算，再分领域排序，避免资源错配
- 保留25%令牌时，模型性能仍保持98.7%以上
- 适合需要高效多模态压缩的部署场景
OmniLLMs中的令牌压缩通常被简化为单一的重要性排序问题：对每个多模态令牌打分，保留前K个。我们指出这种抽象存在根本性偏差——同一注意力分数同时决定了各模态的容量分配和模态内令牌选择。共享的top-K规则继承了音频优先的分配偏见，导致视频令牌尚未竞争就已耗尽资源。为此，我们提出Macer，一种无需训练的压缩器：先为音频和视频分别设定显式预算，再在各模态内部特定浅层进行归一化排序。Macer显著降低令牌开销，同时在音频依赖、音视频联合、视觉主导及视频为中心的基准上均保持精度。在25%保留率下，Qwen2.5-Omni-7B性能保留98.7%，Qwen2.5-Omni-3B保留97.3%。在同为25%保留率时，其表现达到OmniZip水平，且仅需更低计算量。在OmniVinci-9B上，该方法相较共享top-K提升高达12.9点。
原文摘要 · Abstract (English)
Token compression in OmniLLMs is typically posed as a single saliency-ranking problem: score each multimodal token, keep the top-K. We argue this abstraction is mis-specified. The same attention score simultaneously decides two things: how much retained capacity each modality receives, and which tokens within a modality are kept. A shared top-K rule therefore inherits this audio-favoring allocation prior, spending retained capacity on audio before video tokens have a chance to compete. We propose Macer, a training-free compressor that first assigns explicit audio and video budgets, then performs allocation-normalized ranking within each modality at modality-specific shallow layers. Macer significantly reduces token cost while preserving accuracy across audio-grounded, audio--video joint, visual-dominant, and video-centric benchmarks. At 25 % retention, Macer preserves 98.7 % of full-token performance on Qwen2.5-Omni-7B and 97.3 % on Qwen2.5-Omni-3B. On Qwen2.5-Omni-7B, this 25 % setting reaches OmniZip-level performance at 45 % retention while using lower FLOPs. On OmniVinci-9B, the same allocation-before-ranking principle improves over shared top-K ranking by up to 12.9 points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。