arXiv:2601.09496cs.IR2026-01中稿 · SIGIR 2026被引 3

用梯度多子空间微调统一搜索与推荐,提升效率与效果。

Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

  • 将共享与任务特定优化信号分解到互补低秩子空间,减少梯度冲突。
  • 通过正交投影约束参数更新,保护通用知识不被细调数据干扰。
  • 在多个基准数据集上超越现有方法,适用于大规模平台应用。

搜索与推荐(S&R)是在线平台的核心功能,分别应对用户的显式意图和隐式行为。两者互补性推动了统一建模的发展。早期方法采用共享编码器加任务特异性头部,近期研究则将两者排序任务统一为条件生成,可实现端到端优化并利用大模型的语义理解能力。然而,现有方法依赖全量微调,计算成本高且难以扩展。参数高效微调(PEFT)虽更实用,但在统一S&R时面临两大挑战:(1)任务间梯度冲突,源于优化目标差异;(2)因过拟合细调数据导致用户意图理解偏移,破坏通用知识,削弱大模型推理能力。为此,我们提出梯度多子空间微调(GEMS),一种在保留通用知识的同时统一建模搜索与推荐的新框架。GEMS引入(1)多子空间分解,将共享与任务特定优化信号分离至互补低秩子空间,缓解破坏性梯度干扰;(2)零空间投影,限制参数更新位于与通用知识空间正交的子空间内,减轻意图理解偏移。在多个基准数据集上的大量实验表明,GEMS在搜索与推荐任务中均持续优于现有最优基线,表现出更优的性能。

原文摘要 · Abstract (English)

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.

搜索推荐大模型微调参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。