用首个词的概率分布,一眼识破模型生成的套话内容。
Do Stop Me Now: Detecting Boilerplate Responses with a Single Iteration
- 通过分析第一个词的概率分布,快速判断后续是否为套话回复。
- 在小模型到推理型大模型上均实现高准确率识别。
- 可立即终止无效生成,节省算力,适合部署优化场景。
大型语言模型常耗费大量计算资源生成诸如拒绝回应、简单致谢和随意问候等套话内容,增加不必要的成本与延迟。为解决此效率问题,我们提出一种仅需一次生成步骤即可检测此类响应的简单而高效方法。实验表明,首个生成词的对数概率分布能有效表征整个响应的性质。在多种小型、大型及专用推理模型上,不同响应类型的首词对数概率向量形成明显可分的聚类。借助轻量级k-NN分类器,可高精度预测响应是否为实质性回答或套话形式(包括用户自定义拒绝)。该方法提供了一种计算开销极低的实用技术,支持推理过程中的早期终止或切换至小模型,显著降低计算成本。本研究为更高效、可持续的LLM部署提供了直接路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often expend significant computational resources generating boilerplate responses, such as refusals, simple acknowledgements and casual greetings, which adds unnecessary cost and latency. To address this inefficiency, we propose a simple yet highly effective method for detecting such responses after only a single generation step. We demonstrate that the log-probability distribution of the first generated token serves as a powerful signal for classifying the nature of the entire subsequent response. Our experiments, conducted across a diverse range of small, large, and reasoning-specialized models, show that the first-token log-probability vectors form distinctly separable clusters for different response types. Using a lightweight k-NN classifier, we achieve high accuracy in predicting whether a response will be a substantive answer or a form of boilerplate response, including user-specified refusals. The primary implication is a practical, computationally trivial technique, optimizing LLM inference by enabling early termination or redirection to a smaller model, thereby yielding significant savings in computational cost. This work presents a direct path toward more efficient and sustainable LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。