用轻量代理模型+残差高斯点,实现空中视角城市实时渲染。
CityGo: Lightweight Urban Modeling and Rendering with Proxy Buildings and Residual Gaussians
- 混合代理网格与残差高斯点,兼顾精度与效率。
- 训练速度提升1.4倍,移动端可实时渲染大规模城市场景。
- 适合移动AR导航、无人机巡检等边缘设备应用。
大规模城市场景的精确高效建模对AR导航、无人机巡检和智慧数字孪生等应用至关重要。尽管航拍影像覆盖广,但受遮挡、几何不完整和内存消耗高等因素制约,从航拍视角重建城市仍具挑战。近期3D高斯点云(3DGS)虽提升可扩展性和视觉质量,但受限于密集原始体使用、训练时间长及难以部署到边缘设备。本文提出CityGo,一种结合纹理化代理几何与残差及周边3D高斯点的混合框架,实现从航拍视角的轻量化、逼真渲染。首先从多视图立体(MVS)点云提取紧凑建筑代理网格;再利用零阶球谐(zero order SH)高斯点通过图像渲染与逆投影生成无遮挡纹理;为捕捉高频细节,引入基于代理-照片差异并受深度先验指导的残差高斯点;通过重要性感知下采样降低非关键区域冗余,以周围高斯点表示更广阔的城市背景。设计了联合优化策略,同步精炼代理纹理与高斯参数,在移动端GPU上实现复杂城市场景的实时渲染,显著降低训练与内存开销。在真实航拍数据集上的实验表明,该方法平均训练速度提升1.4倍,视觉保真度与纯3DGS相当,且可在消费级移动GPU上实现实时渲染,内存与能耗大幅下降。
原文摘要 · Abstract (English)
Accurate and efficient modeling of large-scale urban scenes is critical for applications such as AR navigation, UAV based inspection, and smart city digital twins. While aerial imagery offers broad coverage and complements limitations of ground-based data, reconstructing city-scale environments from such views remains challenging due to occlusions, incomplete geometry, and high memory demands. Recent advances like 3D Gaussian Splatting (3DGS) improve scalability and visual quality but remain limited by dense primitive usage, long training times, and poor suit ability for edge devices. We propose CityGo, a hybrid framework that combines textured proxy geometry with residual and surrounding 3D Gaussians for lightweight, photorealistic rendering of urban scenes from aerial perspectives. Our approach first extracts compact building proxy meshes from MVS point clouds, then uses zero order SH Gaussians to generate occlusion-free textures via image-based rendering and back-projection. To capture high-frequency details, we introduce residual Gaussians placed based on proxy-photo discrepancies and guided by depth priors. Broader urban context is represented by surrounding Gaussians, with importance-aware downsampling applied to non-critical regions to reduce redundancy. A tailored optimization strategy jointly refines proxy textures and Gaussian parameters, enabling real-time rendering of complex urban scenes on mobile GPUs with significantly reduced training and memory requirements. Extensive experiments on real-world aerial datasets demonstrate that our hybrid representation significantly reduces training time, achieving on average 1.4x speedup, while delivering comparable visual fidelity to pure 3D Gaussian Splatting approaches. Furthermore, CityGo enables real-time rendering of large-scale urban scenes on mobile consumer GPUs, with substantially reduced memory usage and energy consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。