构建统一智能平台，让模型跨任务通用
AI Pangaea: Unifying Intelligence Islands for Adapting Myriad Tasks
- 将多模态数据统一编码，通过296个数据集预训练积累通用知识
- 在45项通用任务和15项科学任务上展现强泛化能力
- 揭示模态扩展的规模效应，为通用智能提供新方向
追求通用人工智能需实现单一模型对海量未见任务的泛化能力。然而当前AI模型局限于特定任务，被称为'智能孤岛'。为此，我们提出Pangaea——首个类地质学泛大陆的AI超级大陆。Pangaea将任意数据编码为统一格式，并在涵盖多种模态的296个数据集上进行预训练，累积通用知识。最终，它在45项通用任务和15项科学任务（覆盖广泛科学领域）中展现出显著泛化性能。深入研究发现，模态扩展具有可量化的规模效应，其知识累积符合几何分布的累积分布函数。整体而言，Pangaea展现出处理海量任务的强大潜力，指明了通向人工智能通用性的新路径。
原文摘要 · Abstract (English)
The pursuit of artificial general intelligence continuously demands generalization in one model across myriad tasks, even those not seen before. However, current AI models are isolated from each other for being limited to specific tasks, now first defined as Intelligence Islands. To unify Intelligence Islands into one, we propose Pangaea, the first AI supercontinent akin to the geological Pangaea. Pangaea encodes any data into a unified format and accumulates universal knowledge through pre-training on 296 datasets across diverse modalities. Eventually, it demonstrates remarkable generalization across 45 general tasks and 15 scientific tasks encompassing a wide range of scientific subjects. By investigating Pangaea deeper, the scaling effect of modality is revealed, quantifying the universal knowledge accumulation across modalities as the cumulative distribution function of a geometric distribution. On the whole, Pangaea shows strong potential to handle myriad tasks, indicating a new direction toward artificial general intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。