重构32个文本游戏代码,提升LLM生成交互世界的能力与可扩展性。
ByteSized32Refactored: Towards an Extensible Interactive Text Games Corpus for LLM World Modeling and Evaluation
- 构建通用基础库,将32个游戏共用逻辑抽象为7个模块,代码量减半。
- 用GPT-4o测试生成新场景游戏,质量在两项指标提升,两项下降。
- 适合研究大模型环境建模、可扩展文本游戏系统的设计者。
模拟交互式世界模型仍是大语言模型(LLMs)的核心挑战。本文提出ByteSized32Refactored,对原始的ByteSized32语料库进行重构,打造一个模块化、可扩展的文本游戏实现。我们优化了每个游戏的代码结构,创建GameBasic.py基础库,将7个基础类(如GameObject)抽象为可复用模块,使总代码量从20,000行降至10,000行。该设计提升了可扩展性:通过共享逻辑,可更高效地引入新场景和规则。基于GPT-4o的实验表明,生成未见场景的游戏在四项评估维度中有两项质量提升,两项下降,说明重构后的分层结构给大模型带来新挑战。总体而言,以基础库和模块化为核心的可扩展架构,不仅促进大模型适应环境规范,也为未来扩展提供可伸缩环境。
原文摘要 · Abstract (English)
Simulating interactive world models remains a core challenge in Large Language Models(LLMs). In this work, we introduce the ByteSized32Refactored, a refactored, modular, and extensible implementation of the original ByteSized32 corpus to explore the task of text game generation. We further optimize the code structure of each text game and create the GameBasic.py foundation library, which centralizes common logic across all 32 games by abstracting 7 base classes (GameObject, etc.) into reusable modules, thereby reducing from 20k to 10k total lines of Python code compared to the original Bytesized32. Our refactored implementation enables extendability - with our centralized design, ByteSized32Refactored can be more efficiently extended to include text games of new scenarios and specifications by reusing the shared logic and functionalities. Extensive experiments with GPT-4o demonstrate a mix of performance - with Bytesized32Refactored, the generated text games for unseen scenarios showcase quality improvements on two of the four evaluation dimensions while decreases on the other two, indicating that the hierarchical structure of the refactored code presents new challenges for LLMs. Overall, we highlight that our extensible code structure, centered on the foundation library and the modular optimization, not only facilitates LLM adaptation to environment specifications but also establishes a scalable environment that supports future extensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。