语言模型在简单图搜索任务中因过度监督而失效，本文提出解决方案使其可解。
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More
- 通过减少监督信号缓解模型依赖捷径的病理
- 在 $D$ 臂星形图上，模型准确率提升至接近最优
- 适合研究大模型推理机制与训练偏差的学者
本文研究路径-星形任务，一种最小化的图搜索范例。图 $G$ 为星形结构，包含 $D$ 条从起点 $s$ 发出的臂，目标节点 $t$ 位于其中一条臂末端。语言模型需根据 $G$、$s$ 和 $t$ 的位置，生成包含 $t$ 的臂。该任务仅需做一次选择：从 $D$ 条臂中选出正确的一条。然而，解码器型语言模型（LM）因学习到吸收训练监督的捷径，在 $D$ 条臂时准确率仅达 $1/D$，无法超越随机猜测。本文揭示此问题源于过度监督，并提出一系列解决方案，证明该任务可通过解码器型语言模型解决。研究发现，任务的极简性导致其难以分解，从而引发困难。这些方法为理解基于下一个词预测训练的语言模型的病理提供了洞见。
原文摘要 · Abstract (English)
This work concerns the path-star task, a minimal example of searching over a graph. The graph, $G$, is star-shaped with $D$ arms radiating from a start node, $s$. A language model (LM) is given $G$, $s$, and a target node $t$, which ends one of the arms and is tasked with generating the arm containing $t$. The minimal nature of this task means only a single choice needs to be made: which of the $D$ arms contains $t$? Decoder-only LMs fail to solve this elementary task above $1/D$ chance due to a learned shortcut that absorbs training supervision. We show how this pathology is caused by excess supervision and we present a series of solutions demonstrating that the task is solvable via decoder-only LMs. We find that the task's minimal nature causes its difficulty, as it prevents task decomposition. Our solutions provide insight into the pathology and its implications for LMs trained via next-token prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。