通过时空令牌筛选提升扩散模型推理准确率，无需训练
ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding
- 利用泰勒预测和视觉注意力质量动态筛选不稳定的文本令牌
- 在多个基准上实现最高9%的准确率提升，且无额外计算开销
- 适合希望改进扩散类多模态模型推理效率的研究者
视觉语言模型（VLMs）虽能通过思维链提示实现强推理，但存在生成成本高、错误累积和自修正能力弱的问题。扩散多模态大语言模型（dMLLMs）以无序方式解码令牌，提升了效率并支持迭代优化，但其推理机制及增强方法仍不明确。本文提出一种无需训练的方法——时空令牌筛选（ST-Veto），利用每一步扩散中可观察所有令牌位置的优势。该方法不仅依赖当前步置信度，还通过二阶泰勒预测置信度动态变化来剔除时间上不稳定的令牌，并结合图像注意力质量过滤弱关联令牌，替换为更可靠的候选。在多个dMLLM和多模态推理基准上，ST-Veto始终优于标准解码策略和已有VLM推理方法，准确率最高提升9%，且不增加训练或生成成本。分析表明，该方法引导生成走向更高置信度、更强视觉对齐的路径。
原文摘要 · Abstract (English)
Vision Language Models (VLMs) achieve strong reasoning with Chain-of-Thought (CoT) prompting but incur high sequential-generation cost, error accumulation, and limited self-correction. Diffusion Multimodal Large Language Models (dMLLMs) unmask tokens in an order-agnostic process, improving efficiency and enabling iterative refinement, yet their reasoning and how to enhance it remain underexplored. We propose a training-free method, Spatio-Temporal Token Veto (ST-Veto), which leverages the ability to observe all token positions at each diffusion step. Rather than relying only on current-step confidence, ST-Veto vetoes temporally unstable tokens via second-order Taylor prediction of confidence dynamics and filters weakly grounded tokens using image-attention mass, swapping them with safer candidates. Across multiple dMLLMs and multimodal reasoning benchmarks, ST-Veto consistently outperforms standard decoding policies and prior VLM reasoning methods, improving accuracy by up to 9% with no additional training or generation cost. Analyses show that ST-Veto steers generation toward higher-confidence, better-grounded paths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。