让大模型在复杂规则中学习，发现适度混沌最能激发智能。
Intelligence at the Edge of Chaos
- 用元胞自动机生成不同复杂度规则，训练大模型预测
- 中等复杂度规则下，模型推理与棋步预测能力最强
- 适合对智能涌现机制感兴趣的学者研究
我们通过研究规则系统复杂度如何影响基于规则的模型预测能力，探索人工系统中智能行为的涌现。研究聚焦于一维元胞自动机（ECA），这类简单但强大的系统可产生从平凡到高度复杂的多种行为。通过在不同ECA上训练不同的大语言模型（LLMs），评估其在下游任务中的表现，发现规则复杂度越高，模型展现出的智能越强，体现在推理和国际象棋走法预测任务上的优异表现。而均匀和周期性系统，以及高度混沌系统，均导致下游性能较差，表明存在一个利于智能形成的复杂度‘甜点’。我们推测，智能源于对复杂性的预测能力，而只需暴露于复杂性即可催生智能。
原文摘要 · Abstract (English)
We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the rules' behavior and the intelligence exhibited by the LLMs, as reflected in their performance on downstream tasks. Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。