用思考次数判断任务难度，提升大模型路由效率并识别恶意提示。
Harnessing Chain-of-Thought Metadata for Task Routing and Adversarial Prompt Detection
- 通过思考次数衡量任务难度，动态分配合适模型。
- 量化小模型路由数学题，延迟降低2%。
- 可精准检测注入攻击，准确率达95%。
本文提出一种名为思考次数（Number of Thoughts, NofT）的指标，用于在预提示阶段判断任务难度，支持大语言模型在生产环境中的高效部署。基于思考次数设定阈值，该指标能有效区分提示难度，实现更优的提示路由。实验显示，将MathInstruct数据集中的提示通过参数量为17亿、70亿和140亿的量化蒸馏版Deepseek模型路由时，延迟降低2%。此外，该指标可用于检测提示注入类对抗性攻击，结合分类器可实现95%的检测准确率。相关实验数据集已公开于GitHub：https://github.com/rymarinelli/Number_Of_Thoughts/tree/main。
原文摘要 · Abstract (English)
In this work, we propose a metric called Number of Thoughts (NofT) to determine the difficulty of tasks pre-prompting and support Large Language Models (LLMs) in production contexts. By setting thresholds based on the number of thoughts, this metric can discern the difficulty of prompts and support more effective prompt routing. A 2% decrease in latency is achieved when routing prompts from the MathInstruct dataset through quantized, distilled versions of Deepseek with 1.7 billion, 7 billion, and 14 billion parameters. Moreover, this metric can be used to detect adversarial prompts used in prompt injection attacks with high efficacy. The Number of Thoughts can inform a classifier that achieves 95% accuracy in adversarial prompt detection. Our experiments ad datasets used are available on our GitHub page: https://github.com/rymarinelli/Number_Of_Thoughts/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。