无需标注或模型权重,用自监督扩散机制让AI自动学习编剧技能。
Training Skills Like Parameters via Self-Supervised Semantic Diffusion

- 模仿扩散模型,通过对比生成与原作差异来自我进化
- 在短剧剧本生成任务中显著提升专业能力,效果超越单一检查点
- 适合想构建自主学习智能体的研究者与开发者
尽管大型语言模型具备出色的指令遵循能力,但在创意剧本写作等高度专业化、开放性的领域仍难以媲美人类专家。现有方法多依赖微调或强化学习,但需访问模型权重且计算开销大,所学知识也难以人工审视。近期代理持续学习方法尝试通过积累外部文本技能弥补差距,但严重依赖昂贵的人工标注或不可靠的LLM评分反馈。为此,我们提出一种受扩散模型启发的无监督自演化代理框架,利用高质量人类作品构建自监督信号。训练过程遵循神经网络经典流程:前向传播、损失计算、反向传播,损失来自代理生成结果与原始人类作品的对比。更新对象不是模型权重,而是外部文本技能库。在短剧剧本生成任务上的实验表明,该方法使代理能自主提取并内化高度通用的技能,显著增强其领域生成能力。自对比反思机制为智能体提供了一条无需外部监督即可自我学习复杂高质量人类成果的可扩展路径。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) demonstrate remarkable general instruction-following capabilities, they often fall short of human experts in highly specialized, open-ended domains such as creative screenwriting. Prior approaches typically adopt post-training, yet both supervised fine-tuning and reinforcement learning require weight access that closed-source frontier models do not offer, and demand heavy compute. Moreover, what is learned is tied to a single checkpoint and cannot be inspected by humans. Recent advancements in agentic continual learning instead attempt to bridge this gap by accumulating external textual skills. However, these methods heavily rely on costly human expert annotations or unreliable LLM-as-a-judge feedback for reflection. To overcome this bottleneck, we propose a novel, unsupervised self-evolving agent framework inspired by the corruption-and-reconstruction paradigm of diffusion models. Instead of relying on explicit external scoring, we leverage existing high-quality human artifacts to construct self-supervised signals. Training then follows the familiar loop of neural network training, forward, loss, and backward, with the loss coming from contrasting the agent's reconstruction against the human original. What is updated is not model weights but an external library of textual skills. We evaluate our framework on the challenging task of short drama screenwriting. Experimental results demonstrate that our method enables the agent to autonomously extract and internalize highly generalizable skills, significantly enhancing its domain-specific generation capabilities. Furthermore, this self-contrastive reflection paradigm offers a scalable pathway for agents to teach themselves the production of complex, high-quality human artifacts, without requiring external supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。