用结构化模板让小模型高效处理长文本推理。
PRISM: Efficient Long-Range Reasoning With Short-Context LLMs
- 用结构化模式引导短上下文大模型进行推理。
- 上下文长度缩短4倍，计算成本降低54%。
- 无需训练即可适配新任务，适合资源受限场景。
长序列任务需要对长输入进行推理，但现有方法受限：长上下文模型需大量算力，参数高效微调（PEFT）依赖训练数据，检索增强生成（RAG）需复杂任务定制。尽管上下文学习方法克服了部分问题，但基于短上下文大模型的方法效率低，以牺牲上下文为代价处理更多标记。我们提出PRISM，一种基于结构化模式的高令牌效率上下文学习方法，在多种任务上表现优于基线，上下文长度缩短4倍。该方法生成简洁输出，并有效利用键值缓存，使成本最高降低54%。PRISM可缩至极短上下文而无额外开销或质量损失，通过从任务描述生成模式，可轻松泛化至新任务。
原文摘要 · Abstract (English)
Long-range tasks demand reasoning over long inputs. However, existing solutions are limited, e.g., long-context models require large compute budgets, parameter-efficient fine-tuning (PEFT) needs training data, and retrieval-augmented generation (RAG) entails complex task-specific designs. Though in-context approaches overcome many of these issues, methods with short-context LLMs are inefficient, trading context for processing more tokens. We introduce PRISM, a highly token-efficient in-context method based on structured schemas that outperforms baselines on diverse tasks with 4x shorter contexts. This approach produces concise outputs and efficiently leverages key-value (KV) caches to reduce costs by up to 54%. PRISM scales down to tiny contexts without increasing costs or sacrificing quality, and generalizes to new tasks with minimal effort by generating schemas from task descriptions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。