让大模型自己判断输出质量，提升推理效率
IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluation
- 用自省标记在预填充阶段自评输出质量
- 在问答任务上达到90%的预测准确率，比外部模型高14%
- 适合需要高效调度的大模型系统使用
大语言模型在生成结果时难以预判输出质量。现有方法依赖外部分类器（如DeBERTa），存在上下文长度受限、表征能力弱和额外计算开销等问题。本文提出IntroLM，通过引入仅在自省标记激活的条件LoRA，在不改变原模型行为的前提下，使因果语言模型在预填充阶段即可自我评估输出质量。在Qwen3 8B上，该方法在问答基准上实现90%的ROC AUC，优于DeBERTa分类器14个百分点。集成至多模型路由系统后，可在保持可靠性前提下，将延迟降低最高达33%，大模型使用量减少最高50%。
原文摘要 · Abstract (English)
A major challenge for the operation of large language models (LLMs) is how to predict whether a specific LLM will produce sufficiently high-quality output for a given query. Existing approaches rely on external classifiers, most commonly BERT based models, which suffer from limited context windows, constrained representational capacity, and additional computational overhead. We propose IntroLM, a method that enables causal language models to predict their own output quality during the prefilling phase without affecting generation using introspective tokens. By introducing token conditional LoRA that activates only for the introspective token, the model learns to predict the output quality for a given query while preserving the original backbone behavior and avoiding external evaluators. On question answering benchmarks, IntroLM applied to Qwen3 8B achieves a ROC AUC of 90 precent for success prediction, outperforming a DeBERTa classifier by 14 precent. When integrated into multi model routing systems, IntroLM achieves superior cost performance tradeoffs, reducing latency by up to 33 precent and large model usage by up to 50 precent at matched reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。