arXiv:2608.21628cs.SEcs.RO2026-08

用AI代理自动探索3D应用,生成可复现的测试证据库。

ExploreAI: Agentic Exploration Knowledge Bases for Reproducible Observable-Regression Testing of Black-Box VR and 3D Applications

论文配图:ExploreAI: Agentic Exploration Knowledge Bases for Reproducible Observable-Regression Testing of Black-Box VR and 3D Applications
图 1 · 摘自论文原文
  • 让大模型规划探索路径,控制导航与多视角拍摄
  • 构建结构化知识库,记录每个物体的发现证据与验证结果
  • 提升测试复现效率,适合自动化测试与质量保障团队

黑盒虚拟现实与3D应用难以进行回归测试,因可观察故障依赖于测试者位置、可见物体及视图捕捉。手动探索能发现故障,但复现耗时;系统性扫描可复现,却缺乏语义引导且浪费探索资源。本文提出ExploreAI,一种基于大模型的智能代理框架,将感知、导航、多视角拍摄与日志记录交由专用模块执行,而由大模型负责规划、证据记录、拍摄策略与验证决策。该框架构建探索知识库(EKB),以结构化形式记录单次探索中每个物体的扫描证据、目标选择、导航路径、多视角图像及自验证结果。在Unity中的六个室内外场景(AI2-THOR、BeamNG)测试显示,ExploreAI在完整与目标探索下均生成高完备性EKB;消融实验表明语义规划、拍摄策略、证据记录与自验证均有贡献。复现实验进一步证明,基于EKB的轨迹能更有效地帮助人类和大模型复现精确的物体-视图证据。

原文摘要 · Abstract (English)

Black-box VR and 3D applications are difficult to regression test because observable failures depend on where a tester moves, what objects are visible, and which views are captured. Manual exploratory testing can find such failures, but its evidence is time-consuming to reproduce; systematic sweeps are reproducible, but they lack semantic guidance and spend exploration budget on low-value viewpoints. We observe that an LLM can make the high-level decisions a human tester makes during exploration: interpreting a task, choosing which objects to inspect, grouping related objects, recording what it saw, and deciding when missing evidence should trigger another attempt. Based on this observation, we present ExploreAI, an LLM-driven agentic framework that offloads repeated perception, navigation, multi-view capture execution, and logging to specialized modules while using the LLM for planning, evidence recording, capture-policy decisions, and verification decisions. ExploreAI constructs an Exploration Knowledge Base (EKB): a structured, per-object record of one exploration run. For each object the agent finds, the EKB stores the scan evidence that exposed it, the selected target, the navigation path, the multi-view capture, and the self-verification result. The EKB is a reusable testing artifact that supports reproducible observable-regression checking across versions of a VR or 3D application. Across six indoor and outdoor scenes in Unity, AI2-THOR, and BeamNG, ExploreAI constructs high-completeness EKBs under both complete and target exploration, and an LLM-module ablation shows where semantic planning, capture policy, evidence recording, and self-verification contribute. Reproduction pilots further show that EKB-guided traces help both humans and LLM-based reproducers reproduce exact object-view evidence more effectively than conditions without EKB context.

AI测试3D测试可复现性智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。