提出无需插值的三平面提升方法,实现嵌入式设备高效3D感知。
TriLift: Interpolation-Free Tri-Plane Lifting for Efficient 3D Perception on Embedded Systems
- 直接投影3D体素至平面特征,通过广播求和重建特征体
- 在分类与补全任务上保持或提升精度,计算量大幅降低
- 适合资源受限的嵌入式机器人系统实时推理
稠密3D卷积虽精度高,但计算开销大,难以用于实时机器人系统。现有三平面方法依赖2D图像特征、插值、点查询和隐式MLP,计算负担重且不适用于嵌入式3D推理。为此,我们提出TriLift,一种无插值的三平面提升与体素融合框架,直接将3D体素投影到平面特征,并通过广播与求和重建特征体。该设计将非线性转移至2D卷积,降低复杂度且完全并行化。为缓解投影导致的空间信息损失,引入轻量级自适应位置编码模块,动态恢复细粒度几何细节,开销极小。为捕捉全局上下文,增加低分辨率体素分支,通过轻量级融合层与提升特征结合,实现高效端到端GPU加速。在分类、补全、分割和检测任务上验证效果,结果表明分类与补全任务精度保持或提升,分割与检测存在精度-效率权衡,计算需求显著降低,仅小幅损失精度。在NVIDIA Jetson Orin Nano上的本地基准测试证实其具备鲁棒的实时吞吐能力,适用于嵌入式机器人感知。
原文摘要 · Abstract (English)
Dense 3D convolutions provide high accuracy for perception but are too computationally expensive for real-time robotic systems. Existing tri-plane methods rely on 2D image features with interpolation, point-wise queries, and implicit MLPs, which makes them computationally heavy and unsuitable for embedded 3D inference. As an alternative, we propose TriLift, a novel interpolation-free tri-plane lifting and volumetric fusion framework that directly projects 3D voxels into plane features and reconstructs a feature volume through broadcast and summation. This shifts nonlinearity to 2D convolutions, reducing complexity while remaining fully parallelizable. To mitigate spatial information loss inherent in projections, we incorporate a lightweight adaptive positional encoding module that helps bridge the spatial information gap, dynamically recovering fine geometric details with negligible overhead. To capture global context, we add a low-resolution volumetric branch fused with the lifted features through a lightweight integration layer, yielding a design that is both efficient and end-to-end GPU-accelerated. To validate the effectiveness of the proposed method, we conduct experiments on classification, completion, segmentation, and detection, and we map the trade-off between efficiency and accuracy across tasks. Results show that classification and completion retain or improve accuracy, while segmentation and detection show a trade-off, significantly reducing computational demand with only a slight decrease in accuracy. On-device benchmarks on an NVIDIA Jetson Orin Nano confirm robust real-time throughput, demonstrating the suitability of the approach for embedded robotic perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。