arXiv:2605.16638cs.AI2026-05

用隐式思考令牌加速多模态推理表示,效果更好且推理成本不变。

TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens

论文配图:TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens
图 1 · 摘自论文原文
  • 用潜变量替代显式思维链,通过优化隐式思考令牌生成推理路径。
  • 在MMEB-v2上性能超越显式思维链模型,推理成本恒定。
  • 思考令牌可解释,适合需要高效推理的多模态应用。

近期研究发现,通用多模态嵌入(UME)能显著受益于思维链(CoT)推理。在此范式中,生成模型为多模态查询生成显式推理轨迹,最终表示从关注查询和推理的<eos>嵌入令牌中提取。尽管有效,生成显式思维链的计算开销常难以承受。本文提出用隐式思考令牌替代显式思维链,这些令牌作为潜在变量可生成显式思维链作为观测变量。通过使用思维链生成损失优化思考令牌,再用对比损失优化后续嵌入令牌,我们在保持恒定推理成本的前提下,实现了高性能的推理感知表示。研究探讨了两个关键架构设计:1)思考令牌与嵌入令牌如何从同一LLM主干中提取;2)令牌如何作为两个相关任务进行训练。我们提出TTE-Flash-2B,一个推理感知的多模态表示模型,在MMEB-v2基准上优于其显式思维链对应模型,同时生成的隐式思考令牌在文本和视觉上均具可解释性。此外,对15个视频数据集的零样本评估显示,随着思考令牌数量增加,性能呈可扩展趋势,并启发了一项基于任务需求自适应分配思考预算的初步研究。

原文摘要 · Abstract (English)

Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradigm, a generative model produces explicit reasoning traces for a multimodal query, with the final representation extracted from an <eos> embedding token attending to both the query and the reasoning. Despite its effectiveness, the computational overhead of generating explicit CoT traces is often prohibitive. In this work, we propose replacing explicit CoT with latent think tokens, which are interpreted as latent variables that can produce explicit CoT traces as observed variables. By optimizing think tokens using CoT generation loss and subsequent embedding tokens using contrastive loss, we produce high-performance, reasoning-aware representations at a constant inference cost. Our study investigates two key architectural designs: 1) how think and embeddings tokens should be extracted from the same LLM backbone. 2) how the tokens should be trained as two dependent tasks. We introduce TTE-Flash-2B, a reasoning-aware multimodal representation model that outperforms its explicit-CoT counterpart on the MMEB-v2 benchmark, while producing latent think tokens that are interpretable both textually and visually. Furthermore, zero-shot evaluation across 15 video datasets reveals scaling behavior as the number of think tokens increases, and motivating a pilot study of adaptive think budget allocation based on task requirements.

多模态推理增强高效生成隐式表征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。