将深度学习计算嵌入像素级，实现低功耗实时医疗影像分析
A Pathway to Near Tissue Computing through Processing-in-CTIA Pixels for Biomedical Applications
- 在电容式跨阻放大像素中集成计算功能，实现像素级深度学习推理
- 在1280x1024图像上降低12倍数据带宽，分割准确率仅下降1.3%-2.5%
- 适合需要低延迟、高能效的手术导航与实时诊断场景
近组织计算需要传感器级别对高分辨率图像进行处理，这对实时生物医学诊断和手术引导至关重要。为此，我们提出一种基于电容式跨阻放大器的像素内计算（CTIA-IPC）架构。该设计利用广泛用于生物医学成像的CTIA像素，其具备优异线性度、低噪声和弱光下稳定工作等优势。我们在像素中引入像素内计算（IPC），实现多通道、多比特卷积运算，并在模数转换器外围集成了批量归一化（BN）和修正线性单元（ReLU）功能。该设计提升了乘累加（MAC）操作的线性度并增强计算效率。通过3D集成技术嵌入像素电路与权重存储，保持了与标准CTIA设计相当的像素密度。算法与电路协同设计，实现高效实时诊断与AI医疗分析。在EndoVis组织数据集（1280x1024）上，CTIA-IPC实现约12倍数据带宽降低，部件分割交并比达75.91%，器械分割达28.58%，较基线方法仅损失1.3%-2.5%准确率。达到1.98 GOPS吞吐量和3.39 GOPS/W能效，为生物医学近组织计算提供高效框架。
原文摘要 · Abstract (English)
Near-tissue computing requires sensor-level processing of high-resolution images, essential for real-time biomedical diagnostics and surgical guidance. To address this need, we introduce a novel Capacitive Transimpedance Amplifier-based In-Pixel Computing (CTIA-IPC) architecture. Our design leverages CTIA pixels that are widely used for biomedical imaging owing to the inherent advantages of excellent linearity, low noise, and robust operation under low-light conditions. We augment CTIA pixels with IPC to enable precise deep learning computations including multi-channel, multi-bit convolution operations along with integrated batch normalization (BN) and Rectified Linear Unit (ReLU) functionalities in the peripheral ADC (Analog to Digital Converters). This design improves the linearity of Multiply and Accumulate (MAC) operations while enhancing computational efficiency. Leveraging 3D integration to embed pixel circuitry and weight storage, CTIA-IPC maintains pixel density comparable to standard CTIA designs. Moreover, our algorithm-circuit co-design approach enables efficient real-time diagnostics and AI-driven medical analysis. Evaluated on the EndoVis tissu dataset (1280x1024), CTIA-IPC achieves approximately 12x reduction in data bandwidth, yielding segmentation IoUs of 75.91% (parts), and 28.58% (instrument)-a minimal accuracy reduction (1.3%-2.5%) compared to baseline methods. Achieving 1.98 GOPS throughput and 3.39 GOPS/W efficiency, our CTIA-IPC architecture offers a promising computational framework tailored specifically for biomedical near-tissue computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。