让两个语言模型通过隐藏状态实时协作，提升算术与逻辑任务准确率。
The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models
- 用可训练接口连接两模型隐藏层，实现双向实时通信。
- 算术任务准确率从36%升至96%，逻辑题效率提高1.7倍。
- 无需文本串行，适合需要工具协同的复杂推理场景。
现有多模型与工具增强系统依赖文本生成进行通信，所有交互均通过输出词汇表串行传递。能否让两个预训练语言模型通过连续、并行的通道进行协调？本文提出双腔模型（Bicameral Model），通过可训练神经接口耦合两个冻结的语言模型的中间隐藏状态。在每一步生成中，主模型驱动任务，辅助模型处理工具、求解约束或执行代码，双方通过翻译网络和学习到的抑制门（占总参数约1%）相互条件激活。该门由任务损失自动学习选择性通信协议，无需预设格式。我们在三种工具后端上验证了该机制：在算术任务中，将两个0.5B模型与计算器耦合，准确率从36%提升至96%；在逻辑网格谜题中，两个0.6B模型与Z3求解器结合，在ZebraLogic数据集上表现达未增强基线的1.7倍；在数学推理中，辅助模型仅凭隐藏状态信号即可生成问题相关代码，无需直接看到问题文本。
原文摘要 · Abstract (English)
Existing multi-model and tool-augmented systems communicate by generating text, serializing every exchange through the output vocabulary. Can two pretrained language models instead coordinate through a continuous, concurrent channel? The Bicameral Model couples two frozen language models through a trainable neural interface on their intermediate hidden states. At every generation step, both models run in lockstep: a primary model drives the task while an auxiliary model operates tools, solves constraints, or executes code, with both conditioning on each other's activations through a translation network and a learned suppression gate ($\sim$1\% of combined parameters). The gate learns a selective communication protocol from task loss alone, without a prescribed format. We demonstrate the mechanism across three tool backends. On arithmetic, coupling two 0.5B models with a calculator raises accuracy from 36\% to 96\%. On logic grid puzzles, coupling two 0.6B models with a Z3 solver achieves $1.7\times$ the unaugmented baseline on ZebraLogic. On mathematical reasoning, coupling with a Python sandbox enables the auxiliary to generate problem-specific code from hidden-state signals alone, without ever seeing the problem text.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。