用提示工程让大模型生成龙与地下城游戏指令，发现指令细节决定成败。
Does Reasoning Help LLM Agents Play Dungeons and Dragons? A Prompt Engineering Experiment
- 通过精心设计提示词引导模型生成符合规则的聊天机器人命令
- 指令微调仅改一句话就显著影响输出准确性，最佳效果达87%成功率
- 普通指令型模型已足够，无需复杂推理模型，适合游戏自动化开发
本文研究大语言模型（LLMs）在《龙与地下城》（DnD）游戏中预测玩家行为并将其转化为Avrae Discord机器人命令的应用。基于FIREBALL数据集，我们评估了推理型模型DeepSeek-R1-Distill-LLaMA-8B和指令型模型LLaMA-3.1-8B-Instruct在命令生成任务中的表现。结果表明，向模型提供具体指令至关重要——仅调整提示词中的一句话，就能显著改变输出质量；且指令型模型在该任务中表现已足以满足需求，无需依赖复杂的推理模型。研究强调了提示工程在实际应用中的关键作用。
原文摘要 · Abstract (English)
This paper explores the application of Large Language Models (LLMs) and reasoning to predict Dungeons & Dragons (DnD) player actions and format them as Avrae Discord bot commands. Using the FIREBALL dataset, we evaluated a reasoning model, DeepSeek-R1-Distill-LLaMA-8B, and an instruct model, LLaMA-3.1-8B-Instruct, for command generation. Our findings highlight the importance of providing specific instructions to models, that even single sentence changes in prompts can greatly affect the output of models, and that instruct models are sufficient for this task compared to reasoning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。