用强化学习让3D高斯点云自动调节密度，提升渲染质量与效率
Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting
- 将密度控制转为可学习的策略网络，取代人工规则
- 在多个数据集上实现更优重建质量，计算复杂度从O(N²)降至O(N)
- 适合需要高质量3D重建且追求高效渲染的研究者
尽管3D高斯点云（3DGS）展现出出色的实时渲染性能，其效果仍受限于对启发式密度控制的依赖。尽管已有诸多针对这些手工规则的改进，但这类方法本质上缺乏适应复杂几何场景的灵活性。本文提出一种范式转变：将密度控制从固定启发式规则转向完全可学习的策略。我们提出LeGS框架，将密度控制重构为参数化策略网络，并通过强化学习进行优化。核心在于基于敏感性分析设计的精准奖励函数，能准确衡量单个高斯分布对重建质量的边际贡献。为保持计算可行性，我们推导出闭式解，将奖励计算复杂度从O(N²)降低至O(N)。在Mip-NeRF 360、Tanks & Temples和Deep Blending数据集上的大量实验表明，LeGS显著优于现有最优方法，在重建质量与效率之间取得更优平衡。代码将于https://github.com/AaronNZH/LeGS发布。
原文摘要 · Abstract (English)
While 3D Gaussian Splatting (3DGS) has demonstrated impressive real-time rendering performance, its efficacy remains constrained by a reliance on heuristic density control. Despite numerous refinements to these handcrafted rules, such methods inherently lack the flexibility to adapt to diverse scenes with complex geometries. In this paper, we propose a paradigm shift for density control from rigid heuristics to fully learnable policies. Specifically, we introduce \textbf{LeGS}, a framework that reformulates density control as a parameterized policy network optimized via Reinforcement Learning (RL). Central to our approach is the tailored effective reward function grounded in sensitivity analysis, which precisely quantifies the marginal contribution of individual Gaussians to reconstruction quality. To maintain computational tractability, we derive a closed-form solution that reduces the complexity of reward calculation from $O(N^2)$ to $O(N)$. Extensive experiments on the Mip-NeRF 360, Tanks \& Temples, and Deep Blending datasets demonstrate that \textbf{LeGS} significantly outperforms state-of-the-art methods, striking a superior balance between reconstruction quality and efficiency. The code will be released at https://github.com/AaronNZH/LeGS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。