arXiv:2602.18981cs.AI2026-02

仅用像素实现游戏内导航,验证视觉模型在复杂关卡中的可行性

How Far Can We Go with Pixels Alone? A Pilot Study on Screen-Only Navigation in Commercial 3D ARPGs

  • 基于视觉显著点构建纯视觉导航代理,无地图或路径信息
  • 在类《黑暗之魂》关卡中成功通行大部分必经路段
  • 揭示当前视觉模型在真实复杂场景下的局限性,适合研究游戏理解

现代3D游戏关卡高度依赖视觉引导,但关卡可导航性的量化仍具挑战。以往研究或在简化环境中模拟游玩,或分析静态截图的视觉可操作性,均无法真实反映玩家在复杂真实游戏关卡中的探索行为。本文基于现有开源视觉可操作性检测器,构建了一个仅依赖屏幕画面的探索与导航代理,该代理接收实时游戏帧,识别显著兴趣点,并通过最小动作空间的有限状态控制器,在类《黑暗之魂》的线性关卡中尝试抵达目标区域。初步实验表明,该代理能通行多数必要段落,展现出有意义的视觉导航行为;但也暴露了底层视觉模型的局限,难以实现全面可靠的自动导航。我们认为该系统为复杂游戏中视觉导航提供了具体、共享的基准与评估协议,并呼吁更多关注此必要任务。结果表明,仅依赖视觉感知、无显式推理的离散单模态模型,在理想化环境下可有效支持导航与环境理解,但自身难成通用解法。

原文摘要 · Abstract (English)

Modern 3D game levels rely heavily on visual guidance, yet the navigability of level layouts remains difficult to quantify. Prior work either simulates play in simplified environments or analyzes static screenshots for visual affordances, but neither setting faithfully captures how players explore complex, real-world game levels. In this paper, we build on an existing open-source visual affordance detector and instantiate a screen-only exploration and navigation agent that operates purely from visual affordances. Our agent consumes live game frames, identifies salient interest points, and drives a simple finite-state controller over a minimal action space to explore Dark Souls-style linear levels and attempt to reach expected goal regions. Pilot experiments show that the agent can traverse most required segments and exhibits meaningful visual navigation behavior, but also highlight that limitations of the underlying visual model prevent truly comprehensive and reliable auto-navigation. We argue that this system provides a concrete, shared baseline and evaluation protocol for visual navigation in complex games, and we call for more attention to this necessary task. Our results suggest that purely vision-based sense-making models, with discrete single-modality inputs and without explicit reasoning, can effectively support navigation and environment understanding in idealized settings, but are unlikely to be a general solution on their own.

视觉导航游戏智能单模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。