arXiv:2603.26765cs.AI2026-03

用位板技术提速53倍,打造高效可扩展的强化学习棋盘游戏框架

Bitboard version of Tetris AI

  • 采用位板表示法,通过位运算加速碰撞检测与消行计算
  • 新算法在10x10网格上3分钟内达平均3829分,性能超越传统方法
  • 适合追求高效训练和低延迟模拟的研究者用于序列决策研究

游戏引擎效率与策略优化算法对复杂序列决策任务(如俄罗斯方块)中强化学习(RL)智能体的训练至关重要。现有俄罗斯方块实现存在仿真速度慢、状态评估不佳和训练范式低效等问题,限制了其在大规模强化学习研究中的应用。本文提出一种基于位板优化与改进强化学习算法的高性能俄罗斯方块AI框架。首先,我们使用位板表示法重设计游戏棋盘与方块,利用位运算加速核心流程(如碰撞检测、消行处理及Dellacherie-Thiery特征提取),相较OpenAI Gym-Tetris提升53倍速度。其次,引入后状态评估的演员网络,利用俄罗斯方块的后状态特性简化价值估计,参数更少但表现更优。第三,提出一种缓冲优化的近端策略优化(PPO)算法,兼顾采样与更新效率,在10x10网格上3分钟内平均得分为3,829。此外,开发了符合OpenAI Gym标准的Python-Java接口,支持与现代强化学习框架无缝集成。实验表明,该框架通过底层位板优化与高层智能策略结合,显著提升了俄罗斯方块作为强化学习基准的实用性,提供了一种样本高效且计算轻量的可扩展序列决策解决方案。

原文摘要 · Abstract (English)

The efficiency of game engines and policy optimization algorithms is crucial for training reinforcement learning (RL) agents in complex sequential decision-making tasks, such as Tetris. Existing Tetris implementations suffer from low simulation speeds, suboptimal state evaluation, and inefficient training paradigms, limiting their utility for large-scale RL research. To address these limitations, this paper proposes a high-performance Tetris AI framework based on bitboard optimization and improved RL algorithms. First, we redesign the Tetris game board and tetrominoes using bitboard representations, leveraging bitwise operations to accelerate core processes (e.g., collision detection, line clearing, and Dellacherie-Thiery Features extraction) and achieve a 53-fold speedup compared to OpenAI Gym-Tetris. Second, we introduce an afterstate-evaluating actor network that simplifies state value estimation by leveraging Tetris afterstate property, outperforming traditional action-value networks with fewer parameters. Third, we propose a buffer-optimized Proximal Policy Optimization (PPO) algorithm that balances sampling and update efficiency, achieving an average score of 3,829 on 10x10 grids within 3 minutes. Additionally, we develop a Python-Java interface compliant with the OpenAI Gym standard, enabling seamless integration with modern RL frameworks. Experimental results demonstrate that our framework enhances Tetris's utility as an RL benchmark by bridging low-level bitboard optimizations with high-level AI strategies, providing a sample-efficient and computationally lightweight solution for scalable sequential decision-making research.

强化学习位板优化游戏AI高效模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。