用关键词引导的级联机制，省钱又保质生成文本。
KiC: Keyword-inspired Cascade for Cost-Efficient Text Generation with LLMs
- 通过关键词匹配筛选弱模型输出中最代表性的答案
- 根据语义一致性判断是否升级强模型，平均节省28.81%成本
- 适合追求低成本高精度文本生成的研究与应用
大型语言模型在自然语言处理任务中表现优异，但高性能模型通常仅可通过API访问，导致推理成本高昂。级联方法通过先使用低成本模型，必要时再升级到更强模型来缓解此问题。然而，现有方法依赖精确文本匹配，难以可靠选择代表性输出或评估自由格式输出的整体可信度。为此，我们提出关键词引导级联（KiC），一种面向低成本自由文本生成的新框架。KiC从弱模型多个输出中识别最具代表性的答案，并评估其他响应与该答案的语义对齐程度。基于对齐度决定是否接受弱模型输出或升级至强模型。在三个自由文本生成基准上的实验表明，KiC在平均减少28.81% API成本的同时，达到GPT-4 97.53%的准确率，且在某一基准上甚至优于GPT-4。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated state-of-the-art performance across a wide range of natural language processing tasks. However, high-performing models are typically accessible only via APIs, incurring substantial inference costs. Cascade methods address this by initially employing a cheaper model and escalating to a stronger one only when necessary. Nevertheless, existing cascade approaches struggle to select a reliable representative response and assess the overall reliability of free-form outputs, as they rely on exact text matching. To overcome these limitations, we propose Keyword-inspired Cascade (KiC), a novel framework for cost-efficient free-form text generation. KiC identifies the most representative answer among multiple outputs from a weaker model and evaluates the semantic alignment of other responses with it. Based on the degree of alignment, KiC determines whether to accept the weaker model's output or escalate to a stronger model. Experiments on three free-form text generation benchmarks show that KiC achieves 97.53 percent of GPT-4's accuracy while reducing API costs by 28.81 percent on average, and even outperforms GPT-4 in a specific benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。