arXiv:2504.04711cs.AI2025-04

AI自主生成数据并用真实指标训练,突破人类知识限制。

Generalising from Self-Produced Data: Model Training Beyond Human Constraints

  • AI通过环境交互自动生成策略和代码,以磁盘空间等真实指标为奖励。
  • 在无监督条件下实现模型自我重训与持续泛化,避免模型崩溃。
  • 适合研究自主智能、自进化系统或超越人类标注的场景。

当前大语言模型受限于人类生成的数据,并受单一抽象层级制约,难以进行确定性真理判断。本文提出一种新框架,让AI模型通过与环境直接交互,自主生成并验证新知识。核心是使用无限且不可操纵的数值奖励(如磁盘空间占用或粉丝数)引导学习,无需人类标注基准。AI代理迭代生成策略和可执行代码,以最大化该指标,成功结果用于自我重训和渐进式泛化。为缓解模型坍塌与冷启动问题,框架强调实证验证而非文本相似度,并支持通过GRPO进行微调。系统采用模块化代理架构,分别负责环境分析、策略生成和代码合成,支持可扩展实验。本工作为迈向自进化AI系统提供了路径,使模型能突破人类施加的约束,向自主通用智能演进。

原文摘要 · Abstract (English)

Current large language models (LLMs) are constrained by human-derived training data and limited by a single level of abstraction that impedes definitive truth judgments. This paper introduces a novel framework in which AI models autonomously generate and validate new knowledge through direct interaction with their environment. Central to this approach is an unbounded, ungamable numeric reward - such as annexed disk space or follower count - that guides learning without requiring human benchmarks. AI agents iteratively generate strategies and executable code to maximize this metric, with successful outcomes forming the basis for self-retraining and incremental generalisation. To mitigate model collapse and the warm start problem, the framework emphasizes empirical validation over textual similarity and supports fine-tuning via GRPO. The system architecture employs modular agents for environment analysis, strategy generation, and code synthesis, enabling scalable experimentation. This work outlines a pathway toward self-improving AI systems capable of advancing beyond human-imposed constraints toward autonomous general intelligence.

自主智能自训练强化学习通用智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。