arXiv:2503.01906cs.LGcs.AI2025-03被引 2

用全局工作区思想让模型分步推理,高效完成加法任务。

Learning to Chain Operations by Routing Information Through a Global Workspace

  • 通过门控机制在模块间传递信息,实现分步操作链式处理。
  • 在未见过的加法运算上表现优于LSTM和Transformer,支持内外推。
  • 适合需要逻辑推理的轻量级模型设计,如教育或低资源场景。

我们提出一种受全局工作区理论启发的模型,通过专用模块协作完成序列推理任务。控制器使用门控机制,将信息有选择地在模块间传递,实现系统性推理。我们在一个简单的加法任务上进行评估:两个加数需依次经输入、增量(多次)、输出模块处理。首先采用人工设计的模块处理独热编码数字,控制器为LSTM网络,学习正确模块调用顺序;其次,用学习的表示模块替代手写模块,对MNIST图像进行处理,增量模块也基于任务目标训练。结果显示,尽管参数更少,该模型在未见加法运算(包括插值与外推)上仍优于LSTM和Transformer,验证了基于全局工作区理论架构在提升深度学习推理能力上的潜力。

原文摘要 · Abstract (English)

We present a model inspired by the Global Workspace Theory that integrates specialized modules to perform a sequential reasoning task. A controller selectively routes information between modules through the workspace using a gating mechanism. This approach allows the model to chain operations by iteratively broadcasting information between specialized domains, mimicking System-2 reasoning. We evaluate the model's performance on a simple addition task, where two addends must be summed. The task can be solved by routing information sequentially through an Input module, an Increment module (multiple times), and finally an Output module. We consider two implementations of this system with increasing complexity. First, using hand-designed modules operating on one-hot digit representations, the controller (a LSTM recurrent network) learns to select the appropriate modules (input, increment, output) in the appropriate sequence. Second, we replace the hand-designed modules with learned representation modules for MNIST images and an increment module trained on the task objectives; here again, the controller learns the appropriate sequential module selection to solve the task. Finally, we show that the Global Workspace model, while having fewer parameters, outperforms LSTMs and Transformers when tested on unseen addition operations (both interpolations and extrapolations of addition operations seen during training). Our results highlight the potential of architectures inspired by the Global Workspace Theory to enhance deep learning's reasoning capabilities.

推理模型模块化序列任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。