将大模型的偏好能力蒸馏到嵌入系统，实现高效个性化图文检索
Embedding the Teacher: Distilling vLLM Preferences for Scalable Image Retrieval
- 用大模型的偏好排序信息训练嵌入模型，保留语义对齐能力
- 在个性化推荐任务中超越传统嵌入方法，提升检索准确率
- 适合需要高精度个性化检索的电商、内容平台场景
文本-图像检索在商品推荐等应用中至关重要。基于嵌入的方法（如CLIP）通过向量相似性搜索实现大规模高效检索，但主要在字面化图文对上训练，难以捕捉商品推荐中常见的抽象或人物驱动属性（如“送给爱园艺的母亲的礼物”）。相比之下，先进的视觉-语言模型（vLLM）能灵活对齐文本与图像，但受限于上下文长度，无法直接处理大规模目录检索。本文提出一种框架，将强大vLLM的偏好排序知识蒸馏到嵌入系统中，在保持嵌入方法推理效率的同时，迁移其精细对齐能力。在人物驱动的商品推荐任务上实验表明，该方法显著优于现有嵌入基线，为个性化文本-图像检索提供了高效解决方案。
原文摘要 · Abstract (English)
Text--image retrieval is necessary for applications such as product recommendation. Embedding-based approaches like CLIP enable efficient large-scale retrieval via vector similarity search, but they are primarily trained on literal caption-like text--image pairs and often fail to capture abstract or persona-driven attributes common in product recommendation applications (e.g., ``a gift for a mother who loves gardening''). In contrast, state-of-the-art vision--language models (vLLMs) can align text with images in a flexible manner, but their limited context window prevents them from directly handling retrieval over large catalogs. We propose a framework that distills the preference rankings of a powerful vLLM into an embedding-based system, transferring its nuanced alignment abilities while maintaining the inference-time scalability of an embedding-based approach. Experiments on persona-driven product recommendation tasks demonstrate that our method significantly outperforms existing embedding-based baselines, providing an efficient solution for personalized text--image retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。