用输入输出轨迹训练神经计算机，实现可学习的计算与交互
Neural Computers
- 将传统计算机的计算、内存和输入输出统一为可学习的运行态
- 仅从输入输出轨迹中学会基础界面操作，如输入对齐和短期控制
- 适合对神经计算新范式感兴趣的研究人员
我们提出一个新方向：神经计算机（NCs），将传统计算机的计算、内存和输入输出统一在可学习的运行态中。长期目标是实现完全神经计算机（CNC）——这一新兴机器形式的成熟通用版本，具备稳定执行、显式重编程和持久能力复用。作为初步探索，我们研究仅从收集的输入输出轨迹中（无需程序状态监控）能否学习到基础的NC原语。具体地，我们将NC实例化为视频模型，在命令行（CLI）和图形界面（GUI）场景下，从指令、像素和用户动作（若有）中逐帧生成屏幕内容。结果表明，NC能够习得基本界面原语，尤其是输入对齐和短时控制；但常规复用、可控更新和符号稳定性仍具挑战。我们提出了迈向CNC的路线图，旨在建立超越当前智能体和传统计算机的新计算范式。
原文摘要 · Abstract (English)
We propose a new frontier: Neural Computers (NCs) that unify computation, memory, and I/O of traditional computers in a learned runtime state. Our long-term goal is the Completely Neural Computer (CNC): the mature, general-purpose realization of this emerging machine form, with stable execution, explicit reprogramming, and durable capability reuse. As an initial step, we study whether elementary NC primitives can be learned solely from collected I/O traces, without instrumented program state. Concretely, we instantiate NCs as video models that roll out screen frames from instructions, pixels, and user actions (when available) in CLI and GUI settings. We show that NCs can acquire elementary interface primitives, especially I/O alignment and short-horizon control, while routine reuse, controlled updates, and symbolic stability remain challenging. We outline a roadmap toward CNCs, to establish a new computing paradigm beyond today's agents and conventional computers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。