通过重构上下文生成新谜题，提升大模型解谜能力
RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation
- 自动构建上下文相关的谜题增强示例
- 在少样本设置下显著提升垂直与横向思维表现
- 适合需要强化逻辑推理的AI系统研究者
解谜需要高级推理能力，推动大语言模型进行抽象思考和创造性求解，常暴露出其认知局限。本文通过多选题形式评估大模型的解谜能力，探究不同提示策略对需多种推理技能谜题的影响。为提升效果，提出RISCORE（基于上下文重构的解谜方法），一种完全自动化的提示技术，通过生成并利用上下文重构的句子级谜题，结合原始示例构建少样本范例。实验表明，RISCORE在各类少样本设置下显著优于传统示例选择策略，有效提升模型在垂直与横向思维任务中的表现。
原文摘要 · Abstract (English)
Riddle-solving requires advanced reasoning skills, pushing LLMs to engage in abstract thinking and creative problem-solving, often revealing limitations in their cognitive abilities. In this paper, we examine the riddle-solving capabilities of LLMs using a multiple-choice format, exploring how different prompting techniques impact performance on riddles that demand diverse reasoning skills. To enhance results, we introduce RISCORE (RIddle Solving with COntext REcontruciton) a novel fully automated prompting method that generates and utilizes contextually reconstructed sentence-based puzzles in conjunction with the original examples to create few-shot exemplars. Our experiments demonstrate that RISCORE significantly improves the performance of language models in both vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。