动态潜码路由让小数据微调效果更好，比传统方法平均高6.6个百分点。
Dynamic Latent Routing
- 用动态搜索联合学习潜码、路由策略和模型参数，一步完成训练。
- 在四个数据集上，小样本微调平均提升6.6个百分点，超越监督微调。
- 能学出有明确因果作用的结构化路由行为，适合高效微调场景。
我们研究了马尔可夫决策过程（MDP）中随时间变化奖励函数下的子策略时序拼接问题。提出通用戴克斯特拉搜索（GDS），并证明通过中间最优子策略的时序组合，可恢复全局最优目标到达策略。受GDS‘搜索、选择、更新’原则启发，提出动态潜码路由（DLR），一种语言模型后训练方法，通过单阶段动态搜索联合学习离散潜码、路由策略与模型参数。在低数据微调设置下，DLR在四个数据集和六种模型上匹配或超越监督微调，平均提升6.6个百分点；而先前的离散潜码基线始终低于SFT。机制分析与针对性代码消融表明，DLR学到了具有明确因果角色的结构化路由行为。
原文摘要 · Abstract (English)
We investigate the temporal concatenation of sub-policies in Markov Decision Processes (MDP) with time-varying reward functions. We introduce General Dijkstra Search (GDS), and prove that globally optimal goal-reaching policies can be recovered through temporal composition of intermediate optimal sub-policies. Motivated by the "search, select, update" principle underlying GDS, we propose Dynamic Latent Routing (DLR), a language-model post-training method that jointly learns discrete latent codes, routing policies, and model parameters through dynamic search in a single training stage. In low-data fine-tuning settings, DLR matches or outperforms supervised fine-tuning across four datasets and six models, achieving a mean gain of +6.6 percentage points, while prior discrete-latent baselines consistently underperform SFT. Mechanistic analyses and targeted code ablations show that DLR learns structured routing behaviors with distinct causal roles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。