用用户互动数据训练多模态商品嵌入，提升广告推荐效果。
CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads Retrieval
- 基于大模型统一表示图文信息，共享嵌入空间。
- 用用户共同点击对优化模型，提升召回率与转化率。
- 可直接替换现有系统，线上显著提升点击和转化。
物品到物品（I2I）检索是大规模推荐与广告系统的核心组件。在 Snap 动态产品广告（DPA）中，传统 I2I 检索面临两个挑战：视觉、文本和多模态编码器分离导致检索链路割裂，且仅依赖内容训练的嵌入无法对齐驱动转化的共同互动行为。本文提出 CAMIE，一种面向 Snap DPA 检索的共互动感知多模态物品嵌入框架。CAMIE 基于 LLM/MLLM 主干模型，利用其原生多模态接口将商品图像与元数据映射到统一嵌入空间，并在用户旅程中挖掘出的共互动物品对上，使用对称的批内 InfoNCE 目标进行微调。离线实验显示，CAMIE 在 Recall@10 上优于最强商业多模态嵌入模型，且能以极小质量损失支持纯文本检索。线上部署中，它作为即插即用替代品，取代两个已上线的内容型 I2I 编码器，相比多模态对照组提升 +0.390% CTR / +10.832% CVR，相比文本对照组提升 +18.958% CTR / +13.12% CVR，整体流量下提升 +0.211% CTR / +1.911% CVR。CAMIE 已投入生产。
原文摘要 · Abstract (English)
Item-to-item (I2I) retrieval is a core primitive in large-scale recommendation and advertising systems. In production Snap Dynamic Product Ads (DPA), I2I retrieval faces two challenges: separate visual, textual, and multimodal encoders fragment the retrieval stack, and content-only training does not align embeddings with the co-engagement behavior that drives downstream conversions. We present CAMIE, a co-engagement-aware multimodal item embedding framework for Snap DPA retrieval. CAMIE builds on LLM/MLLM backbones, using their native multimodal interfaces to represent item images and metadata in a shared embedding space. It then fine-tunes the backbone on co-engaged item pairs mined from user journeys with a symmetric in-batch InfoNCE objective. Offline, CAMIE outperforms the strongest commercial multimodal embedding model on Recall@10 and serves text-only retrieval from the same checkpoint with minimal quality loss. Online, CAMIE serves as a drop-in replacement for two deployed content-based I2I encoders, delivering +0.390% CTR / +10.832% CVR over the multimodal control, +18.958% CTR / +13.12% CVR over the text control, and +0.211% CTR / +1.911% CVR on overall DPA traffic. CAMIE is deployed in production.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。