让模型自适应调整生成掩码的令牌长度，提升分割效率与质量
ALTo: Adaptive-Length Tokenizer for Autoregressive Mask Generation
- 设计可预测令牌长度的机制，动态分配注意力资源
- 在多个分割基准上达到领先性能，且令牌开销可自适应调节
- 适合关注多模态生成效率与精度平衡的研究者
人类在绘制视觉对象和形状时能根据复杂度自适应分配注意力，而现有多模态大语言模型仍受限于固定的令牌表示。为弥合这一差距，我们提出 ALTo，一种用于自回归掩码生成的自适应长度分词器。为此，设计了新颖的令牌长度预测器，引入长度正则化项及可微分的令牌分块策略。进一步构建 ALToLLM，将 ALTo 无缝集成至多模态大语言模型中。通过组相对策略优化（GRPO）实现对掩码质量与效率之间权衡的偏好控制。实验表明，ALToLLM 在多个主流分割基准上取得当前最优性能，且具备自适应令牌成本。代码与模型已开源：https://github.com/yayafengzi/ALToLLM。
原文摘要 · Abstract (English)
While humans effortlessly draw visual objects and shapes by adaptively allocating attention based on their complexity, existing multimodal large language models (MLLMs) remain constrained by rigid token representations. Bridging this gap, we propose ALTo, an adaptive length tokenizer for autoregressive mask generation. To achieve this, a novel token length predictor is designed, along with a length regularization term and a differentiable token chunking strategy. We further build ALToLLM that seamlessly integrates ALTo into MLLM. Preferences on the trade-offs between mask quality and efficiency is implemented by group relative policy optimization (GRPO). Experiments demonstrate that ALToLLM achieves state-of-the-art performance with adaptive token cost on popular segmentation benchmarks. Code and models are released at https://github.com/yayafengzi/ALToLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。