通过扩展训练信号覆盖，显著提升生成式推荐的长序列建模与跨域信息利用能力。
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
- 引入时序对比学习和跨域学习双辅助目标，突破单步预测局限
- 在美团四领域数据集上，点击率提升1.8%，商品交易额增长2.1%
- 方法无推理开销，适合工业级推荐系统部署
Next-Token Prediction（NTP）存在两个结构性训练信号局限：一是仅优化单步预测，忽略长程行为结构的学习（称作时间局部性）；二是多领域序列中，目标项嵌入仅接收前一隐藏状态的梯度，缺乏跨领域上下文的显式梯度路径（称作空间局部性）。本文提出NONTP，通过两个辅助目标扩展NTP的信号覆盖。TCL（时序对比学习）采用BYOL风格的EMA教师模型与InfoNCE，对齐隐藏状态与未来K步轨迹的表示；TDL（跨域学习）对跨域隐藏状态做均值池化，并通过共享预测头进行预测，开辟第二条无额外参数的梯度路径。两者均在推理阶段丢弃，零开销。在美团四领域工业数据集（全排序场景）上，NONTP相比NTP提升HR@10达+34.3%，相比MBGR提升+18.3%；在公开的Amazon Movie-Book-CDs基准上，HR@10提升+2.8%，NDCG@10提升+3.7%。在线A/B测试验证了CTR +1.8%、GMV +2.1%（p < 0.01）。消融实验表明各组件独立贡献，梯度冲突被识别为未来研究方向。
原文摘要 · Abstract (English)
Next-Token Prediction (NTP) carries two structural training signal limitations. First, NTP optimizes for single-step prediction only, placing no supervised pressure on learning longer-range behavioral structure -- we term this \textbf{temporal locality}. Second, in multi-domain sequences, each target item embedding receives gradient updates exclusively from the immediately preceding hidden state, with no explicit gradient pathway from cross-domain context -- we term this \textbf{spatial locality}. We propose \textbf{NONTP}, extending NTP's signal coverage along both dimensions through two auxiliary objectives. \textbf{TCL (Temporal Contrastive Learning)} uses a BYOL-style EMA teacher with InfoNCE to align hidden states against a $K$-step future trajectory in representation space. \textbf{TDL (Trans-Domain Learning)} mean-pools cross-domain hidden states and predicts through the shared prediction head, opening a second gradient pathway with no additional parameters. Both are discarded at inference: zero overhead. On a four-domain Meituan industrial dataset (full ranking), NONTP achieves HR@10 +34.3\% over NTP and +18.3\% over MBGR. On the public Amazon Movie-Book-CDs benchmark, HR@10 +2.8\% and NDCG@10 +3.7\%. Online A/B tests confirm CTR +1.8\% and GMV +2.1\% (both $p < 0.01$). Ablation studies confirm each component contributes independently, with gradient conflict analyzed as a direction for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。