arXiv:2511.20683cs.CL2025-11

根据问题难易自动选回复模板,省钱又不降质。

Dynamic Template Selection for Output Token Generation Optimization: MLP-Based and Transformer Approaches

  • 用MLP或Transformer判断问题复杂度,动态选合适回复模板。
  • 测试中准确率达90.5%,输出token减少32.6%~33.9%。
  • 跨主流大模型通用,适合想降推理成本的开发者。

当前大模型部署普遍对各类问题使用统一提示策略,无论复杂分析还是简单事实问答均采用冗长回复模式,造成显著的令牌浪费。由于输出令牌成本是输入令牌的4-8倍,这一问题尤为突出。本文提出动态模板选择(DTS)机制,根据查询复杂度自适应匹配响应模板,在不牺牲回答质量的前提下实现显著成本降低。对比了两种路由方法:基于预计算嵌入的简单MLP与微调的RoBERTa Transformer。在1,000道MMLU题目上评估,MLP路由准确率达90.5%,略高于RoBERTa的89.5%,且参数量少1.25亿。实证分析显示模板选择具备跨提供商泛化能力——通过9,000次生产级API调用验证,该方法在OpenAI GPT-4、Google Gemini和Anthropic Claude三类主流模型上表现稳定,路由准确率均为90.5%,但令牌减少幅度为32.6%至33.9%,反映不同提供商生成特性差异。本工作包含形式化问题定义、机器学习理论基础、四种算法及其复杂度分析,以及跨生产系统的充分验证。

原文摘要 · Abstract (English)

Contemporary large language model deployments typically employ uniform prompting strategies across diverse query types, applying verbose response patterns to both complex analytical tasks and straightforward factual questions. This one-size-fits-all methodology leads to substantial token inefficiency, a concern amplified by the significant cost differential between input and output tokens--the latter commanding 4-8x higher prices across major providers. We present Dynamic Template Selection (DTS), which adaptively matches response templates to query complexity, achieving significant cost reductions without compromising response quality. We compared two routing approaches: a simple MLP that uses pre-computed embeddings and a more complex fine-tuned RoBERTa transformer. Through comprehensive evaluation on 1,000 MMLU questions, we find that the MLP router achieves 90.5% routing accuracy on held-out test data, marginally exceeding RoBERTa's performance (89.5%) despite utilizing 125M fewer parameters. Notably, our empirical analysis reveals provider-agnostic behavior in template selection--routing decisions generalize effectively across 3 major LLM providers (OpenAI GPT-4, Google Gemini, and Anthropic Claude), as validated through 9,000 production API calls. While routing accuracy remains consistent at 90.5% across providers, observed token reductions vary from 32.6% to 33.9%, reflecting provider-specific generation characteristics. This work contributes several key elements: formal problem formulation with theoretical grounding in machine learning, four algorithms with corresponding complexity analyses, and extensive empirical validation across production systems.

大模型优化提示工程成本控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。