arXiv:2608.20611cs.AI2026-08

针对生成推荐中路径偏离问题,提出动态调整采样策略的优化方法。

Difficulty-Aware Semantic-ID Optimization for Generative Recommendation

论文配图:Difficulty-Aware Semantic-ID Optimization for Generative Recommendation
图 1 · 摘自论文原文
  • 根据候选路径偏离程度动态分配真实补全样本,提升生成连贯性。
  • 在12项指标中11项超越基线,9项达最优,尤其提升深层路径召回率。
  • 适合需要高阶语义路径生成的推荐系统研究与工程落地。

基于语义标识符的生成推荐将检索与排序建模为层次化物品标识符的自回归生成任务。常见训练流程为监督微调(SFT)后接在线策略强化学习(GRPO),但原始GRPO不适应树状结构任务。在冻结SFT检查点下,许多提示的50束约束排序前16个候选中缺失精确目标,困难场景甚至无候选进入目标标识分支。这一诊断揭示训练问题:当在线策略组普遍存在目标缺失时,即使部分候选沿目标路径前进,项目级奖励也可能产生弱或退化的奖励差异。为此,我们提出难度感知语义标识优化(DASO),一种树结构感知的后训练方法,将此失败模式视为在线展开分配问题。DASO通过前缀匹配深度分析每个展开组,定位候选离开目标路径的瓶颈标识层级,再有界地将部分组重新分配至前缀引导的补全样本,同时保留原始展开用于对比。语义标识前缀奖励提供分级奖励,辅助SFT锚点防止对已解决样本的性能退化。在公开基准上,DASO在12项指标中11项优于MiniOneRec风格的GRPO,9项达到最佳;内部推荐任务中也显著提升多数层级召回率。

原文摘要 · Abstract (English)

Semantic-ID-based generative recommendation casts retrieval and ranking as autoregressive generation over hierarchical item identifiers. A common recipe is SFT followed by GRPO, yet vanilla GRPO is poorly matched to this tree-structured task. Under the frozen SFT checkpoint, the exact target is absent from the first 16 candidates of the 50-beam constrained ranking for many prompts, and in harder cases none of these candidates enters the target SID branch. This prompt-level diagnostic motivates a training concern: when on-policy GRPO groups are similarly target-missing, item-level rewards may produce weak or degenerate reward variation even if some candidates follow part of the target path. We propose Difficulty-Aware Semantic-ID Optimization (DASO), a tree-aware post-training method that addresses this failure mode as an online rollout-allocation problem. Instead of using fixed difficulty buckets or uniformly injecting ground-truth completions, DASO profiles each current rollout group by prefix-match depth, locates the bottleneck SID levels where candidates leave the target path, and reallocates a bounded portion of the group to prefix-guided completions while retaining raw rollouts for contrast. A SID-prefix reward provides graded credit, while an auxiliary SFT anchor mitigates regression on examples already solved by the SFT checkpoint. On the public benchmarks, DASO improves over MiniOneRec-style GRPO on 11 of 12 metrics and achieves the best result on 9 of 12 metrics; it also improves most level-wise recall metrics on the internal recommendation task.

生成推荐强化学习路径优化语义标识

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。