直接在倾斜二维投影上实现快速三维大分子检测,无需重建体积数据。
At FullTilt: Real-Time Open-Set 3D Macromolecule Detection Directly from Tilted 2D Projections

- 跳过三维重建,直接在倾斜序列的二维图像上进行端到端检测。
- 推理速度提升数量级,显存占用大幅降低,零样本性能领先。
- 适合需要快速大规模蛋白质组分析的研究者使用。
低温电子断层扫描中的开放集三维大分子检测无需针对特定目标重新训练模型。然而,严格的显存限制导致现有方法只能对提取的子体积进行缓慢的滑动窗口推理,无法处理完整的三维断层图。为此,我们提出 FullTilt,一种端到端框架,通过直接在对齐的二维倾斜序列上操作来重新定义三维检测。由于倾斜序列包含的图像远少于重建断层图中的切片数,FullTilt 消除了冗余的体积分块计算,使推理速度提升数量级。为同时处理整个倾斜序列,我们引入倾斜序列编码器以高效融合多视角信息。此外,我们提出多类别视觉提示编码器实现灵活提示、倾斜感知查询初始化以有效锚定三维查询,并设计辅助几何原型模块,增强模型对多视角几何的理解,同时提高对不良成像伪影的鲁棒性。在三个真实世界数据集上的广泛评估表明,FullTilt 实现了最先进的零样本性能,同时显著降低运行时间和显存需求,为快速、大规模视觉蛋白质组学分析铺平道路。所有代码与数据将在发表后公开。
原文摘要 · Abstract (English)
Open-set 3D macromolecule detection in cryogenic electron tomography eliminates the need for target-specific model retraining. However, strict VRAM constraints prohibit processing an entire 3D tomogram, forcing current methods to rely on slow sliding-window inference over extracted subvolumes. To overcome this, we propose FullTilt, an end-to-end framework that redefines 3D detection by operating directly on aligned 2D tilt-series. Because a tilt-series contains significantly fewer images than slices in a reconstructed tomogram, FullTilt eliminates redundant volumetric computation, accelerating inference by orders of magnitude. To process the entire tilt-series simultaneously, we introduce a tilt-series encoder to efficiently fuse cross-view information. We further propose a multiclass visual prompt encoder for flexible prompting, a tilt-aware query initializer to effectively anchor 3D queries, and an auxiliary geometric primitives module to enhance the model's understanding of multi-view geometry while improving robustness to adverse imaging artifacts. Extensive evaluations on three real-world datasets demonstrate that FullTilt achieves state-of-the-art zero-shot performance while drastically reducing runtime and VRAM requirements, paving the way for rapid, large-scale visual proteomics analysis. All code and data will be publicly available upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。