arXiv:2606.17574cs.AI2026-06被引 3

统一物理AI全栈评估,跨层问题可定位。

DeepInsight: A Unified Evaluation Infrastructure Across the Physical AI Stack

论文配图:DeepInsight: A Unified Evaluation Infrastructure Across the Physical AI Stack
图 1 · 摘自论文原文
  • 用三个统一抽象隔离不同层级差异
  • 单个运行时覆盖从解码到物理控制的跨度
  • 跨层故障可追踪,适合全栈研发团队

物理AI系统评估涵盖跨度超三个数量级的操作——从单次基础模型解码到数千次全身控制物理步进——在模态、奖励语义和资源消耗上差异显著。现有框架无法覆盖此范围,当前依赖拼接多个独立测试工具,各模块无共享运行时与评分机制,虽保局部有效,却丧失跨层诊断所需的统一身份。我们提出DeepInsight,一个可在单一运行时中覆盖全栈的评估基础设施。它不消除异质性,而是通过三个窄抽象(任务、资源、结果)实现统一:一个统一的实验驱动器、一套由所有昂贵后端(包括LLM推理与沙盒环境)实现的资源句柄协议、以及一个统一的事件追踪标识方案。该系统已在具身人形机器人全三层生产部署,新基准仅需配置即可接入。在基础模型层,其复现了已有文献与同行框架的结果,在自身波动范围内,且单节点运行速度更快,多节点扩展近线性。其核心优势在于诊断能力:因各层写入同一追踪日志,跨层回归仍可定位,这是任何分段工具组合无法实现的跨层收益。

原文摘要 · Abstract (English)

Evaluating a Physical AI stack spans operators that differ by more than three orders of magnitude -- from a single foundation-model decoding step to thousands of physics ticks of whole-body control -- varying orthogonally in modality, reward semantics, and resource profile. No existing framework spans this range, so the stack is evaluated today by stitching together separate harnesses that share neither runtime nor scoring, preserving each segment's local validity but losing the shared identity needed to diagnose cross-layer regressions. We present DeepInsight, an evaluation infrastructure that serves this full spectrum on a single runtime. Rather than homogenize the regimes, it preserves their heterogeneity behind three narrow abstractions -- task, resource, and result -- each realized as one invariant shared by every subsystem: one episode driver, one resource-handle protocol implemented by every expensive backend (LLM inference and sandboxed runtimes alike), and one trace identity scheme under which every event is written. Deployed in production across all three layers of an embodied humanoid stack, this single set of invariants onboards new benchmarks largely by configuration. Where mature peer orchestrators exist -- at the foundation-model end -- it reproduces published references and peer-framework readings within their own spread, runs the same suites faster on a single node, and scales near-linearly across nodes. Its distinctive return is diagnostic: because every layer writes into one shared trace, a regression that begins in one layer and surfaces in another stays localizable on that trace -- a cross-layer payoff no federation of per-segment harnesses can reproduce.

评估框架物理AI跨层诊断统一运行时

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。