将NCA模型压缩到胶囊内,实现微型设备上的精准出血检测与深度估计。
eNCApsulate: NCA for Precision Diagnosis on Capsule Endoscopes
- 用大模型伪标签蒸馏训练轻量NCA,适配胶囊内微控制器
- 参数量减少100倍以上,分割精度优于其他小型模型
- 首次在胶囊内实现实时深度估计与出血识别,适合临床诊断
无线胶囊内窥镜是一种无创成像方法,可覆盖整个消化道,是传统内镜的无痛替代方案。其生成的视频数据量大,需大量时间审阅,且摄入后定位胶囊位置困难。出血检测与单目深度估计等技术有助于病灶定位,但深度学习模型通常过大,无法直接在胶囊上运行。本文针对胶囊内窥镜图像,训练用于出血分割和深度估计的神经元胞自动机(NCA)。对于单目深度估计,通过将大型基础模型的输出作为伪真值,对轻量级NCA架构进行知识蒸馏。随后将训练好的NCA部署至ESP32微控制器,实现在仅相机大小的硬件上高效图像处理。相比其他便携式分割模型,NCA在Dice指标上更优,同时内存占用少于100倍。NCA深度估计的视觉结果令人信服,某些情况下甚至超越伪真值的细节与真实感。在ESP32-S3上进行运行时优化,平均推理速度提升超过3倍。通过算法调整与蒸馏策略,成功将NCA模型嵌入可放入无线胶囊内窥镜的微控制器中。这是首个在微型设备上实现可靠出血分割与深度估计的工作,为结合视觉里程计实现胶囊精确定位的精准诊断铺平道路——真正实现‘在胶囊上’的智能分析。
原文摘要 · Abstract (English)
Wireless Capsule Endoscopy is a non-invasive imaging method for the entire gastrointestinal tract, and is a pain-free alternative to traditional endoscopy. It generates extensive video data that requires significant review time, and localizing the capsule after ingestion is a challenge. Techniques like bleeding detection and depth estimation can help with localization of pathologies, but deep learning models are typically too large to run directly on the capsule. Neural Cellular Automata (NCA) for bleeding segmentation and depth estimation are trained on capsule endoscopic images. For monocular depth estimation, we distill a large foundation model into the lean NCA architecture, by treating the outputs of the foundation model as pseudo ground truth. We then port the trained NCA to the ESP32 microcontroller, enabling efficient image processing on hardware as small as a camera capsule. NCA are more accurate (Dice) than other portable segmentation models, while requiring more than 100x fewer parameters stored in memory than other small-scale models. The visual results of NCA depth estimation look convincing, and in some cases beat the realism and detail of the pseudo ground truth. Runtime optimizations on the ESP32-S3 accelerate the average inference speed significantly, by more than factor 3. With several algorithmic adjustments and distillation, it is possible to eNCApsulate NCA models into microcontrollers that fit into wireless capsule endoscopes. This is the first work that enables reliable bleeding segmentation and depth estimation on a miniaturized device, paving the way for precise diagnosis combined with visual odometry as a means of precise localization of the capsule -- on the capsule.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。