用记忆混合机制提升流式3D重建的长期稳定性
MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction
- 将递归状态拆分为多个独立记忆块，仅更新不匹配部分
- 在7-Scenes数据集上平均降低15.3%重建完整度误差（最高40%）
- 无需训练、无额外参数，可直接接入现有模型使用
3D重建是3D视觉的基础任务，也是空间智能的关键能力。尤其在实时空间感知中，流式3D重建至关重要，但现有递归在线模型在长序列下常因状态漂移和遗忘导致性能退化，需推理时修复。我们提出MeMix，一种无需训练、即插即用的模块，通过将递归状态重构为记忆混合体来提升流式重建。MeMix将状态划分为多个独立记忆块，仅更新最不匹配的记忆块，其余保持不变。该选择性更新策略缓解灾难性遗忘，同时维持O(1)推理内存，且无需微调或新增可学习参数，可直接应用于现有递归重建模型。在标准基准（ScanNet、7-Scenes、KITTI等）上，采用相同主干网络与推理设置，MeMix在7-Scenes的300–500帧序列中，平均降低重建完整度误差15.3%（最高达40.0%）。代码已公开于https://dongjiacheng06.github.io/MeMix/
原文摘要 · Abstract (English)
Reconstruction is a fundamental task in 3D vision and a fundamental capability for spatial intelligence. Particularly, streaming 3D reconstruction is central to real-time spatial perception, yet existing recurrent online models often suffer from progressive degradation on long sequences due to state drift and forgetting, motivating inference-time remedies. We present MeMix, a training-free, plug-and-play module that improves streaming reconstruction by recasting the recurrent state into a Memory Mixture. MeMix partitions the state into multiple independent memory patches and updates only the least-aligned memory patches while exactly preserving others. This selective update mitigates catastrophic forgetting while retaining $O(1)$ inference memory, and requires no fine-tuning or additional learnable parameters, making it directly applicable to existing recurrent reconstruction models. Across standard benchmarks (ScanNet, 7-Scenes, KITTI, etc.), under identical backbones and inference settings, MeMix reduces reconstruction completeness error by 15.3% on average (up to 40.0%) across 300--500 frame streams on 7-Scenes. The code is available at https://dongjiacheng06.github.io/MeMix/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。