用可控大模型监控训练过程,自动调参防过拟合与探索失效
AI Training Manager: Bounded Closed-Loop Control of Adaptive Training Recipes

- 基于结构化数据的闭环控制,动态调整学习率等多维参数
- 在TinyStories上降低60%验证损失,干预日志可审计
- 适合需要安全、可解释训练调控的研究者和工程团队
我们提出AI Training Manager,一个基于大模型的受约束监督控制器,用于自适应机器学习训练。标准训练流程常依赖固定配方或单轴调度器,难以应对中段运行中的严重过拟合、损失不平衡、探索坍缩或不安全探索等问题。该管理器不替代数学优化器或作为无约束编码代理,而是通过模式约束接口:读取活跃训练的结构化遥测快照,审核受限动作空间,并返回经验证的训练参数更新,如学习率、正则强度、损失权重系数和探索设置。我们在监督语言建模与强化学习任务中评估该架构。在TinyStories数据集上,管理器检测并纠正过拟合,使验证损失比基线降低60%,同时生成可审计的干预日志。在监督设定下,管理器推断无需阻塞训练循环:训练可继续进行,待验证更新后异步应用。在机器人操控强化学习任务中,相同受限决策接口以分幕闭环方式使用,管理器更新在评估或检查点边界应用。管理器有效缓解了保守与不安全的探索模式。结果表明,模式约束的大模型可作为实时训练的受控监督管理者,为传统优化器与调度器提供可解释、多轴干预能力。
原文摘要 · Abstract (English)
We present the AI Training Manager, a bounded LLM-based supervisory controller for adaptive machine learning training. Standard training pipelines often rely on fixed recipes or single-axis schedulers, which can struggle with mid-run failures such as severe overfitting, loss imbalance, exploration collapse, or unsafe exploration. Rather than replacing mathematical optimizers or acting as an unconstrained coding agent, the manager operates through a schema-conditioned interface: it reads structured telemetry snapshots from an active run, audits a constrained action space, and returns validated updates to training parameters such as learning rate, regularization strength, loss-weight coefficients, and exploration settings. We evaluate this architecture across supervised language modeling and reinforcement learning. On TinyStories, the manager detects and corrects overfitting, achieving a validation loss 60% lower than the baseline while producing auditable intervention logs. In this supervised setting, we additionally show that manager inference does not need to block the training loop: training can continue while a manager response is pending, and validated updates can be applied asynchronously once available. In a robotic manipulation reinforcement-learning task, we use the same bounded decision interface in an episodic closed-loop setting, where manager updates are applied at evaluation or checkpoint boundaries. The manager mitigates both conservative and unsafe exploration regimes. These results suggest that schema-conditioned LLMs can serve as bounded supervisory managers for live training runs, complementing conventional optimizers and schedulers with interpretable, multi-axis intervention capabilities
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。