arXiv:2511.14948cs.CV2025-11被引 4

用自研LED时钟实现多相机毫秒级精准同步,支持可见光与红外设备。

RocSync: Millisecond-Accurate Temporal Synchronization for Heterogeneous Camera Systems

  • 用红光和红外LED编码时间,通过视频帧解码曝光起止时刻。
  • 实测同步误差仅1.34毫秒(RMSE),优于灯光、音频等传统方法。
  • 适用于医疗、工业等无控环境,可提升三维重建与姿态估计效果。

多视角视频流的时空对齐对动态场景应用至关重要,如多视角三维重建、姿态估计与场景理解。然而,在混合使用专业与消费级设备、可见光与红外传感器,或含/不含音频的异构系统中,硬件同步常不可行,尤其在无法控制采集条件的真实环境中更为突出。本文提出一种低成本、通用性强的同步方法,可在多种相机系统间实现毫秒级时间对齐,支持可见光(RGB)与红外(IR)模态。该方法采用自研的「LED Clock」,通过红光与红外LED编码时间信息,使录制帧可被视觉解码出曝光窗口的起止时间。在多组实验中,其残差误差达1.34毫秒(RMSE),优于基于光、音频及时间码的同步方案,并直接提升了多视角姿态估计与三维重建性能。我们在包含25台以上异构相机的大规模手术记录中验证了系统有效性,覆盖红外与可见光模态。该方案简化了同步流程,推动了非受控环境下先进视觉感知的应用落地。

原文摘要 · Abstract (English)

Accurate spatiotemporal alignment of multi-view video streams is essential for a wide range of dynamic-scene applications such as multi-view 3D reconstruction, pose estimation, and scene understanding. However, synchronizing multiple cameras remains a significant challenge, especially in heterogeneous setups combining professional and consumer-grade devices, visible and infrared sensors, or systems with and without audio, where common hardware synchronization capabilities are often unavailable. This limitation is particularly evident in real-world environments, where controlled capture conditions are not feasible. In this work, we present a low-cost, general-purpose synchronization method that achieves millisecond-level temporal alignment across diverse camera systems while supporting both visible (RGB) and infrared (IR) modalities. The proposed solution employs a custom-built \textit{LED Clock} that encodes time through red and infrared LEDs, allowing visual decoding of the exposure window (start and end times) from recorded frames for millisecond-level synchronization. We benchmark our method against hardware synchronization and achieve a residual error of 1.34~ms RMSE across multiple recordings. In further experiments, our method outperforms light-, audio-, and timecode-based synchronization approaches and directly improves downstream computer vision tasks, including multi-view pose estimation and 3D reconstruction. Finally, we validate the system in large-scale surgical recordings involving over 25 heterogeneous cameras spanning both IR and RGB modalities. This solution simplifies and streamlines the synchronization pipeline and expands access to advanced vision-based sensing in unconstrained environments, including industrial and clinical applications.

多相机同步时间对齐手术视觉红外成像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。