通过自适应调节损失平坦度，提升稀疏视角下的细节重建能力。
Do Flat Minima Improve Sparse Novel View Synthesis?
- 根据图像局部结构动态调整损失平坦度正则权重
- 在保持细节重建精度的同时提升泛化性能
- 适用于稀疏视角合成场景的模型优化
尽管近期的新视角合成方法取得了成功，但在稀疏视角设置下仍表现不佳。这种对未见视角的泛化能力差是有限数据训练下的固有问题。我们研究了新视角合成中损失尖锐性与泛化能力的关系——这一方向尚未被充分探索。有趣的是，虽然追求平坦极小值通常能提升深度学习的泛化能力，但在新视角合成中降低损失尖锐性并不总是有益。我们发现，高细节区域需要尖锐的损失景观以实现精确重建，而低细节区域则受益于平坦的损失景观以改善泛化。基于此洞察，我们提出结构感知尖锐性（structure-aware sharpness），在结构自适应邻域内定义，并根据局部图像结构自适应调整尖锐性正则化权重。该策略在保证细节重建所需尖锐性的同时，鼓励更平坦的极小值以增强泛化。在多种数据集和配置下，该方法一致提升了多个基线模型的表现。代码已公开于 https://bbangsik13.github.io/FASR。
原文摘要 · Abstract (English)
Despite the success of recent novel view synthesis methods, they tend to struggle in sparse-view settings. This poor generalization to unseen viewpoints is an inherent challenge when training with limited data. To address this, we investigate the relationship between loss sharpness and generalization in novel view synthesis-an underexplored direction. Interestingly, while pursuing flatter minima is widely known to improve generalization in deep learning, reducing loss sharpness is not always beneficial in novel view synthesis. We demonstrate that this difference arises because high-detail regions inherently require a sharp loss landscape for accurate reconstruction, whereas low-detail regions benefit from a flat loss landscape for improving generalization. Based on this insight, we introduce structure-aware sharpness, defined within structure-adaptive neighborhoods, and propose to adaptively adjust the sharpness regularization weight according to the local image structure. This strategy encourages flatter minima for generalization while preserving the loss sharpness necessary to reconstruct fine details. Across various datasets and configurations, our strategy consistently improves a wide range of baselines. Code is available at https://bbangsik13.github.io/FASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。