加速推理导致多模态生成内容漂移，本文提出控制方法提升一致性。
Faster but Different: Diagnosing and Controlling Content Drift in Accelerated Multimodal Diffusion Language Models
- 通过调整缓存刷新频率，实现速度与生成一致性的平衡
- 在1.3倍加速下达成近似完全一致的输出结果
- 适用于需要稳定生成的部署场景，如内容审核与生产
训练无关的加速使基于扩散的多模态大语言模型（dMLLMs）更易部署，但可能悄然改变生成内容。我们在300张真实图像上研究了这一推理时一致性问题，对比Fast-dLLM与同一模型未加速输出。在长文本设置下（每步提交1.05–1.25个标记），置信度阈值调节虽改变解码行为，但不影响基线一致性。状态刷新消融和图像替换干预表明，陈旧视觉状态与生成文本状态是漂移成因。对于测试的Fast-dLLM实现，缩短KV-cache刷新间隔可形成单调的速度-一致性权衡曲线，在1.3倍加速下实现近乎精确的一致性。该诊断结果也出现在dLLM-Cache和LaViDa中，但dLLM-Cache需同时收紧双缓存才能恢复一致性，从而丧失加速优势。独立提示与图像复现了阈值不敏感与刷新恢复现象。定向审计发现50对低一致性样本中半数存在真实内容替换。在盲评双标注器评估中，加速减基线的事实错误差为0.00（95% CI [-0.17,+0.17]），样本未检测到差异，但无法证明事实等价性。所有测试的自适应或平滑刷新变体在相同计算量下均不及固定间隔。贡献在于一对诊断与实现特定的一致性控制，而非准确率或安全保证。
原文摘要 · Abstract (English)
Training-free acceleration makes diffusion-based multimodal large language models (dMLLMs) more deployable, but it may silently change generated content. We study this serving-time consistency problem on 300 real images, comparing Fast-dLLM outputs with the same model's unaccelerated outputs. Across the mild parallelism induced in our long-form setting (1.05--1.25 committed tokens per step), confidence-threshold tuning changes decoding behavior but not baseline agreement. State-refresh ablations and an image-swap intervention instead identify stale visual and generated-text states as contributors to drift. For the tested Fast-dLLM implementation, shortening the KV-cache refresh interval yields a monotonic speed--agreement frontier and near-exact agreement at a measured 1.3x speedup. The initial diagnosis also appears with dLLM-Cache and LaViDa, although dLLM-Cache recovers agreement only after both caches are tightened, which removes its speed advantage. Independent prompts and images reproduce the threshold-insensitivity and refresh recovery. A targeted audit finds genuine content substitution in half of 50 low-agreement pairs. In a separate blinded two-annotator evaluation, the pooled accelerated-minus-baseline factual-error difference is 0.00 (95% CI [-0.17,+0.17]); this sample detects no difference but does not establish factual equivalence. Finally, none of the tested adaptive or smoothed-refresh variants beats the fixed interval at matched compute. Our contribution is a paired diagnostic and an implementation-scoped consistency control, not an accuracy or safety guarantee.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。