用自监督方式训练判别器，让大模型更快生成文本
SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification
- 用目标模型自身生成标签，自动训练判别器
- 在多个任务上实现更优的推理速度与准确率平衡
- 无需人工标注，适合各类自然语言任务
推测解码通过用小型草稿模型生成候选词，并由大型目标模型验证来加速大语言模型推理。近期的判别解码通过放宽验证标准，接受与目标模型输出存在轻微差异的草稿词，但现有方法受限于对人工标注或可验证真值任务的依赖，难以推广至多样化的自然语言任务。我们提出SelfJudge，通过目标模型的自监督学习训练判别验证器。该方法通过评估替换后词汇是否保持原始回答语义一致来衡量语义保留性，从而实现跨多样化NLP任务的自动验证器训练。实验表明，SelfJudge在推理-精度权衡上优于判别解码基线，提供了一种广泛适用的快速大模型推理方案。
原文摘要 · Abstract (English)
Speculative decoding accelerates LLM inference by verifying candidate tokens from a draft model against a larger target model. Recent judge decoding boosts this process by relaxing verification criteria by accepting draft tokens that may exhibit minor discrepancies from target model output, but existing methods are restricted by their reliance on human annotations or tasks with verifiable ground truths, limiting generalizability across diverse NLP tasks. We propose SelfJudge, which trains judge verifiers via self-supervision of the target model. Our method measures semantic preservation by assessing whether token-substituted responses preserve the meaning of original responses, enabling automatic verifier training across diverse NLP tasks. Our experiments show SelfJudge achieves superior inference-accuracy trade-offs than judge decoding baselines, offering a broadly applicable solution for faster LLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。