优化提示词让小型语言模型高效处理安全查询路由。
Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
- 用提示词工程改进小模型输出格式，无需修改模型权重。
- 微调提示词后，部分小模型得分逼近大模型，最高达95.74。
- 适合资源有限但需安全过滤的部署场景，如边缘计算。
我们研究受控查询路由问题，即用户查询先由路由器判断是否为系统内分布查询，或拒绝可能不安全或超出范围的外部查询。考察紧凑型开源小语言模型（SLMs）在延迟约束下能否同时完成这两项任务。在GQR-Bench上评估22个模型，以分布内与分布外准确率的调和平均数（GQR-Score）评分。结果显示，中等规模的SLMs在低延迟下接近前沿模型的路由性能；但许多紧凑模型因无法稳定遵循输出格式而表现不佳。然而，提示词优化技术可使这些模型在不更改权重的情况下有效应对，显著提升表现。少样本提示优化将Mistral 7B的得分从81.79提升至90.87，使Qwen3.5 9B达到95.74（本研究最优），仅比最强未优化的大模型Gemma 3 27B（96.01）低0.3分。对于Granite 4 Tiny，仅使用裸露DSPy签名（无上下文示例）便使其得分从54.29提升至83.05。结果表明，提示优化是受控查询路由的有效初步策略，但较弱模型仍需权重级适配或模式感知训练。
原文摘要 · Abstract (English)
We study the problem of guarded query routing, where we assume that a user query first meets a router that either determines the ideal endpoint for in-distribution queries or rejects out-of-distribution queries that are potentially unsafe or out of the system's scope. We investigate whether compact open-weight Small Language Models (SLMs) can jointly handle both tasks under latency constraints. We evaluate 22 models on GQR-Bench and score them with the harmonic mean of in-distribution and out-of-distribution accuracy. We find that mid-scale SLMs come close to frontier model routing quality at much lower latency. Still, many compact models fail because they do not reliably follow the required output format. However, our results show that prompt optimization techniques enable SLMs to handle such cases gracefully, without changing the models' weights. Moreover, few-shot prompt optimization raises Mistral 7B from 81.79 to 90.87 GQR-Score and lifts Qwen3.5 9B to 95.74, the best optimized score in our study and within 0.3 points of the strongest unoptimized larger model: Gemma 3 27B at 96.01. The bare DSPy signature, without in-context exemplars, is the most effective strategy for Granite 4 Tiny, raising its score from 54.29 to 83.05. These results show that prompt optimization is a useful first step for guarded query routing, while weaker models may still need weight-level adaptation or schema-aware training
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。