arXiv:2409.06107cs.CLcs.AI2024-09
提出双模块架构，分离大模型生成中的监督信号与有用性能力。
Doppelgänger's Watch: A Split Objective Approach to Large Language Models
- 设计并行于主模型的Doppelgänger模块，实时监督每步生成。
- 同时预测序列到当前步的监督得分，解耦生成与反馈机制。
- 为可控生成提供新思路，适合研究生成控制与模型对齐者。
本文研究大语言模型中的
原文摘要 · Abstract (English)
In this paper, we investigate the problem of "generation supervision" in large language models, and present a novel bicameral architecture to separate supervision signals from their core capability, helpfulness. Doppelgänger, a new module parallel to the underlying language model, supervises the generation of each token, and learns to concurrently predict the supervision score(s) of the sequences up to and including each token. In this work, we present the theoretical findings, and leave the report on experimental results to a forthcoming publication.
大模型生成控制双模块
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。