为分布式声学传感设计流式优先框架,支持实时分析与交互标注
FiLark: a streaming-first software framework for end-to-end exploration, annotation, and algorithm integration in distributed acoustic sensing

- 采用统一数据流抽象,替代传统分段处理方式
- 实现任意长录音的低内存交互浏览与事件标注
- 支持算法无缝集成至可视化流程,适合科研与生产部署
分布式声学传感(DAS)系统以超高的通道数生成连续数据流,其速率超出传统批处理分析框架的能力。因此,长时录音的交互探索、可扩展事件标注及实时算法闭环监控仍难以有效支持,现有工作多依赖手动选取数据片段和离线处理。本文提出FiLark(Fiber Lark),一个基于Python的流式优先框架,将流式原则统一应用于DAS的数据访问、信号处理、可视化与监控。该框架将连续多文件记录视为统一数据流,所有组件围绕此抽象构建。基于OpenGL的环形缓冲渲染器可在恒定内存下交互浏览任意长度数据。内置标注界面支持在连续数据流中直接标注事件,无需离线预处理即可生成可复现的机器学习标注数据集。信号处理库包含时域、空域、频域及分解类算子,提供CPU与PyTorch加速的GPU版本,并支持跨分段边界的状态保持与连续执行。标准化的监控接口可集成流式检测器与学习模型到可视化流程中。通过各层共享统一的流抽象,交互式开发的处理配置可直接迁移至可扩展生产流水线而无需修改。
原文摘要 · Abstract (English)
Distributed acoustic sensing (DAS) systems generate continuous, ultra-high-channel-count data streams at rates that exceed the capabilities of conventional batch-oriented analysis frameworks. As a result, essential tasks such as interactive exploration of long-duration recordings, scalable event annotation, and real-time algorithm-in-the-loop monitoring remain inadequately supported by workflows built around manually selected data segments and offline processing. This paper presents FiLark (Fiber Lark), a Python framework that applies a \emph{streaming-first} principle uniformly across data access, signal processing, visualization and monitoring for DAS. Instead of operating on manually selected data segments, FiLark presents any DAS sources-including continuous multi-file recordings-as a unified stream and builds all system components around that abstraction. An OpenGL-based ring-buffer renderer enables interactive browsing and visualization of arbitrarily long recordings with constant memory usage. An integrated annotation interface supports event labeling directly within continuous data streams, facilitating the creation of reproducible machine-learning-ready labeled datasets without offline preprocessing. The signal processing library includes temporal, spatial, spectral, and decomposition-based operators, with both CPU implementations and GPU-accelerated variants via PyTorch, alongside stateful chunked execution that preserves processing continuity and application semantics across segment boundaries. A standardized monitor interface further integrates streaming detectors and learning-based models into the visualization workflow. By sharing a common streaming abstraction across all layers, FiLark allows processing configurations and workflows developed interactively to transfer directly to scalable production pipelines without modification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。