arXiv:2508.16100cs.CLcs.AI2025-08EMNLP被引 7

无需人工标注,用双模型自循环训练生成指令数据

CYCLE-INSTRUCT: Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency

  • 两个模型互为师生,从无标签文本中自动生成问答对
  • 在四个任务上超越有种子的回译方法,接近强监督效果
  • 适合追求全自动训练、减少人工成本的研究者

指令微调对对齐大语言模型与人类意图至关重要,但现有方法通常依赖昂贵的人工标注种子数据或强大外部教师模型。尽管指令回译技术降低了这一依赖,但仍受限于初始种子集,限制了完全自动化,引入偏差,并导致未标注语料利用效率低下。本文提出Cycle-Instruct,一种实现完全无种子指令微调的新框架。受循环一致性启发,Cycle-Instruct采用双自训练循环,两个模型——答案生成器和问题生成器——仅从原始无标签文本中启动。它们通过对方生成的伪标签重构原文段落,相互监督,从而仅基于数据内在结构学习。我们在四个不同数据赛道(通用指令遵循、领域特定任务、对话日志、普通文本)上验证了其有效性。大量实验表明,Cycle-Instruct不仅优于依赖种子的回译基线,还达到与强监督方法相当的性能。

原文摘要 · Abstract (English)

Instruction tuning is vital for aligning large language models (LLMs) with human intent, but current methods typically rely on costly human-annotated seed data or powerful external teacher models. While instruction back-translation techniques reduce this dependency, they remain fundamentally tethered to an initial seed set, which limits full automation, introduces biases, and can lead to inefficient use of unlabeled corpora. In this paper, we propose Cycle-Instruct, a novel framework that achieves fully seed-free instruction tuning. Inspired by cycle consistency, Cycle-Instruct employs a dual self-training loop where two models-an answer generator and a question generator-are bootstrapped solely from raw, unlabeled text. These models mutually supervise each other by reconstructing original text segments from their counterpart's generated pseudo-labels, effectively learning from the intrinsic structure of the data without any human-provided seeds. We demonstrate Cycle-Instruct's efficacy across four diverse data tracks, including general instruction-following, domain-specific tasks, dialogue logs, and plain text. Our extensive experiments show that Cycle-Instruct not only outperforms seed-driven back-translation baselines but also achieves performance comparable to strongly supervised methods.

指令微调自训练无监督循环一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。