arXiv:2605.25007cs.IR2026-05

用大模型动态路由证据,解决冷启动下多模态推荐缺失问题

Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Reranking

论文配图:Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Reranking
图 1 · 摘自论文原文
  • 将缺失模态视为序列化证据路由问题,用大模型学习最优证据组合策略
  • 在无须人工干预情况下,提升冷启动场景下推荐准确率4.0%以上
  • 适合需要处理多模态数据缺失的推荐系统研发人员参考

多模态推荐系统在冷启动场景中常因用户历史、物品文本或视觉信息缺失而严重失效。现有方法通过插补、特征传播或生成重建恢复缺失信号,但当现存信号较弱时可能引入不实证据。本文提出基于大语言模型的元模态代理(MMA),将缺失视为序列证据路由问题。MMA 在掩码模态样本上通过平衡缺失任务强化学习训练,包含两种变体:MMA-Auto仅使用自动化文本、图像与图工具;MMA-Interactive额外支持基于现存模态的澄清提问,作为上限诊断。MMA在第一阶段检索器生成候选池后进行重排序,而非从全库检索。最终重排融合验证集选定的MMA得分与初始检索得分。评估围绕四个关键验证点展开:无预言机的一模态可用性(OOMA)鲁棒性、各模态的OOMA分解、固定候选池全库重排及确定性路由机制对照。MMA-Auto相比最强非交互基线,在目标正例的OOMA NDCG@10上提升4.0%,在固定候选池全库重排中提升12.7%。不带学习策略的RuleRouter-Fuse表现更差,证明学习型路由优于确定性工具融合。当可提供澄清时,MMA-Interactive进一步获得4.1%的上限增益。

原文摘要 · Abstract (English)

Missing modalities cause severe failures in multimodal recommender systems. User histories, item text, and visual evidence are frequently absent during cold-start scenarios, exactly when recommendation quality matters most. Existing approaches recover absent signals through imputation, feature propagation, or generative reconstruction, but these strategies can inject unsupported evidence when the surviving signals are weak. We introduce the Meta-Modal Agent (MMA), a large language model based candidate-pool reranker that treats missingness as a sequential evidence-routing problem. MMA is trained with balanced missingness-task reinforcement learning over masked-modality episodes and is evaluated in two variants: MMA-Auto, which uses only automated text, image, and graph tools, and MMA-Interactive, which additionally permits clarification questions grounded in surviving modalities as an upper-bound diagnostic. MMA operates after a first-stage retriever has produced a candidate pool; it scores those candidates rather than retrieving items from the full catalog. Final reranking fuses MMA scores with first-stage retrieval scores selected on validation data. Our evaluation is organized around four evidence checks required for a robust missing-modality claim: oracle-free one-observed-modality availability (OOMA) robustness, per-modality OOMA breakdowns, fixed-pool full-catalog reranking, and a deterministic-router mechanism control. MMA-Auto improves target-positive OOMA NDCG@10 by 4.0% and fixed-pool full-catalog reranking NDCG@10 by 12.7% over the strongest non-interactive baseline. RuleRouter-Fuse, which uses the same tools and fusion rule without learned policy updates, underperforms MMA-Auto, supporting learned routing beyond deterministic tool fusion. MMA-Interactive adds a 4.1% upper-bound gain when clarification is available.

多模态推荐冷启动大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。