arXiv:2608.22367cs.CL2026-08中稿 · EMNLP

通过上下文感知的聚类解码,提升多模态大模型长文本生成的连贯性。

Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs

论文配图:Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs
图 1 · 摘自论文原文
  • 基于置信度与邻近度乘积评分,避免过早提交不成熟词汇。
  • 在四个基准上均减少幻觉,长文本生成质量提升显著。
  • 无需训练,适配不同视觉融合策略,适合追求生成质量的研究者。

扩散型多模态大语言模型(dMLLMs)在生成长文本时常出现语义漂移和重复问题,且生成质量随长度增加而下降。我们发现现有解码方法存在两大结构性缺陷:基于置信度的评分忽略已解码邻居的支持,块划分机制阻碍高准备度语义锚点的全局访问,导致词汇在局部上下文未充分建立前就被提交。为此,我们提出 extbf{C}ontext- extbf{A}ware extbf{C}luster extbf{D}ecoding( extit{CACD}),一种无需训练的解码方法。该方法在每个掩码位置采用软最大置信度与邻居邻近度的乘积作为评分,优先选择上下文就绪的词元,抑制低置信度噪声,且无块划分限制,使高准备度锚点全局可及。同时引入架构感知校准,应对不同视觉融合策略带来的置信度异质性。在三个 dMLLM 模型、四个基准上的实验表明,相比原始方法, extit{CACD} 在所有设置中均实现一致的质量提升,并在多个长序列生成任务中表现更优,凸显了邻居支持与视觉融合策略对 dMLLM 解码设计的重要性。代码开源:https://github.com/zhaoyk-sysu/CACD-dMLLM。

原文摘要 · Abstract (English)

Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases. We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors, together causing tokens to be committed before their local context is sufficiently established. We propose \ours{} (\textbf{C}ontext-\textbf{A}ware \textbf{C}luster \textbf{D}ecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity, promoting contextually ready tokens above isolated candidates while suppressing low-confidence positional noise, operating block-free to keep high-readiness anchors globally accessible. \ours{} further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design. Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM.

多模态生成解码优化长文本生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。