解决LLM推荐模型训练难问题,提升效果与稳定性。
Break the Optimization Barrier of LLM-Enhanced Recommenders: A Theoretical Analysis and Practical Framework

- 通过归一化和推荐感知降维,改善嵌入表示优化条件。
- 在多个数据集上实现显著更低的训练损失和更高推荐精度。
- 适合需要稳定高效训练的工业级推荐系统开发者。
大语言模型(LLM)增强的推荐系统将LLM文本表示注入主干模型,以利用丰富的物品文本信息,且无需推理时调用LLM。然而我们发现,现有方法严重阻碍主干模型优化,导致训练损失高且难以降低。为此,我们建立全面的局部优化曲率理论分析,识别出两个关键原因:1)嵌入向量范数差异过大;2)语义-协同结构不匹配的角聚类。基于此,提出轻量级框架TF-LLMER,包含两项核心组件:首先强调物品嵌入归一化的重要性,消除范数驱动的不稳定性,实现可证明的优化条件控制;其次引入Rec-PCA,一种面向推荐任务的降维方法,通过将协同结构注入表示变换,解决语义-协同角聚类错位问题。该方法联合优化语义信息保留与基于交互历史构建的物品共现图对齐,利用图上总变差惩罚促进对齐。理论与大量实验表明,TF-LLMER显著优于现有最先进方法。代码已开源。
原文摘要 · Abstract (English)
Large language model (LLM)-enhanced recommendation models inject LLM representations into backbone recommenders to exploit rich item text without inference-time LLM cost. However, we find that existing LLM-enhanced methods significantly hinder the optimization of backbone models, resulting in high training losses that are difficult to reduce. To address it, we establish a comprehensive theoretical analysis of local optimization curvature and identify two key causes: 1) large norm disparity and 2) semantic-collaboration misaligned angular clustering of LLM representations. Guided by these insights, we propose Training-Friendly LLM-Enhanced Recommender (TF-LLMER), a lightweight framework with two key components. First, we highlight the necessity of item embedding normalization to eliminate norm-driven instability and achieve provable control over optimization conditioning. Second, we introduce Rec-PCA, a recommendation-aware dimensionality reduction method that injects collaborative structure into the representation transformation to resolve semantic-collaboration misaligned angular clustering. It jointly optimizes semantic information retention and alignment with an item-item co-occurrence graph constructed from interaction histories. The graph captures collaborative structure, and alignment is promoted by penalizing total variation over the graph. Both theory and extensive experiments demonstrate that TF-LLMER significantly outperforms state-of-the-art methods. Our code is available at https://github.com/woriazzc/TF-LLMER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。