构建动态游戏RAG评估框架，兼顾内容与社区双重变化。
ChronoPlay: A Framework for Modeling Dual Dynamics and Authenticity in Game RAG Benchmarks
- 双动态更新机制追踪游戏内容与玩家兴趣变化
- 融合官方与玩家社区数据生成真实问答对
- 首个面向游戏场景的持续性RAG基准测试
检索增强生成（RAG）系统在在线游戏等动态领域日益重要，但缺乏专用评估基准制约了其发展。核心挑战在于双重动态：游戏内容更新与玩家社区关注点的持续演变。同时，自动化评估需确保生成问题具备玩家视角的真实性。为此，我们提出ChronoPlay框架，实现游戏RAG基准的自动化与持续生成。该框架采用双动态更新机制追踪两类变化，并通过双源合成引擎结合官方资料与玩家社区数据，保障事实准确性和查询真实性。我们在三款不同游戏中实例化该框架，构建首个面向游戏领域的动态RAG基准，揭示模型在复杂真实条件下的表现。代码已开源：https://github.com/hly1998/ChronoPlay。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) systems are increasingly vital in dynamic domains like online gaming, yet the lack of a dedicated benchmark has impeded standardized evaluation in this area. The core difficulty lies in Dual Dynamics: the constant interplay between game content updates and the shifting focus of the player community. Furthermore, the necessity of automating such a benchmark introduces a critical requirement for player-centric authenticity to ensure generated questions are realistic. To address this integrated challenge, we introduce ChronoPlay, a novel framework for the automated and continuous generation of game RAG benchmarks. ChronoPlay utilizes a dual-dynamic update mechanism to track both forms of change, and a dual-source synthesis engine that draws from official sources and player community to ensure both factual correctness and authentic query patterns. We instantiate our framework on three distinct games to create the first dynamic RAG benchmark for the gaming domain, offering new insights into model performance under these complex and realistic conditions. Code is avaliable at: https://github.com/hly1998/ChronoPlay.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。