通过多次前向传播提升文本嵌入模型的推理能力
Refine Thought: A Test-Time Inference Method for Embedding Model Reasoning
- 在测试时多次运行模型，动态优化语义表示
- 在BRIGHT和PJBenchmark上显著提升推理表现
- 适合需要深度语义理解的任务场景
我们提出RT（Refine Thought），一种增强文本嵌入模型语义推理能力的方法。该方法通过多次前向传播文本嵌入模型，获得最终的语义表示。实验表明，RT在BRIGHT和人岗匹配基准PJBenchmark上取得显著提升，同时在通用语义理解任务如C-MTEB上保持一致性能。结果表明，RT有效激活了编码器-解码器类文本嵌入模型（如Qwen3-Embedding-8B）在预训练中学习到的语义推理能力。RT可视为一种测试时推理方法。
原文摘要 · Abstract (English)
We propose RT (Refine Thought), a method that can enhance the semantic reasoning ability of text embedding models. The method obtains the final semantic representation by running multiple forward passes of the text embedding model. Experiments show that RT achieves significant improvements on semantic reasoning tasks in BRIGHT and the person-job matching benchmark PJBenchmark, while maintaining consistent performance on general-purpose semantic understanding tasks such as C-MTEB. Our results indicate that RT is effective because it further activates the semantic reasoning ability learned during pretraining by decoder-only text embedding models (e.g., Qwen3-Embedding-8B). RT can be seen as a test-time inference method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。