用合成认知模型在序列任务上超越Transformer
Beating Transformers using Synthetic Cognition
- 将合成认知扩展至序列处理，实现上下文感知反应
- 在DNA序列分类任务中优于现有基础模型，胜出更多基准任务
- 适合关注新型认知架构与序列建模的研究者
通向通用人工智能的道路依赖于生成具备上下文感知的即时反应行为，尽管当前变压器（Transformer）架构表现最优，但仍缺乏推理能力。最近提出的合成认知（Synthetic Cognition）新方法已被用于构建即时反应行为。本研究旨在探索其在构建上下文感知反应行为中的应用。我们提出一种针对最新合成认知实现的序列处理机制，并在DNA序列分类任务中与DNA基础模型进行对比。实验结果表明，该方法明显优于现有的DNA基础模型，在更多基准任务中取得最佳成绩。因此，本工作实现了两个目标：将合成认知拓展至序列处理，且在序列分类任务中击败了变压器架构。
原文摘要 · Abstract (English)
The road to Artificial General Intelligence goes through the generation of context-aware reactive behaviors, where the Transformer architecture has been proven to be the state-of-the-art. However, they still fail to develop reasoning. Recently, a novel approach for developing cognitive architectures, called Synthetic Cognition, has been proposed and implemented to develop instantaneous reactive behavior. In this study, we aim to explore the use of Synthetic Cognition to develop context-aware reactive behaviors. We propose a mechanism to deal with sequences for the recent implementation of Synthetic Cognition, and test it against DNA foundation models in DNA sequence classification tasks. In our experiments, our proposal clearly outperforms the DNA foundation models, obtaining the best score on more benchmark tasks than the alternatives. Thus, we achieve two goals: expanding Synthetic Cognition to deal with sequences, and beating the Transformer architecture for sequence classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。