轻量级语音神经假体解码器，内存仅需10GB仍达顶尖性能。
LightBeam: An Accurate and Memory-Efficient CTC Decoder for Speech Neuroprostheses
- 用大模型延迟融合改进束搜索，无需大型语言模型
- 内存占用从320GB降至10GB，性能领先现有方法
- 开源实现，适合临床与研究者部署使用
恢复失语和无言症患者的交流能力，一种有前景的路径是通过脑-文本神经假体，直接从皮层神经活动解码语音。Brain-to-Text '24 和 '25 两个基准发布了来自失语患者的大脑皮层记录，并提供了基于连接时序分类（CTC）训练的基线算法。尽管在这些基准上取得显著进展，所有领先的先前工作都依赖于需要约320 GB内存的WFST-based CTC解码器，严重限制了患者和研究者的可用性。本文提出LightBeam，一种非WFST的CTC解码器，仅需约10 GB内存，在两个基准上均达到当前最优性能。其核心是通过延迟融合将大语言模型集成到束搜索过程中，从而避免对大型N-gram语言模型的依赖。LightBeam采用Python实现，已开源。
原文摘要 · Abstract (English)
A promising pathway for restoring communication in patients with dysarthria and anarthria is speech neuroprostheses, which directly decode speech from cortical neural activity. Two benchmarks, Brain-to-Text '24 and '25, released intracranial recordings from patients with dysarthria along with a baseline algorithm trained with Connectionist Temporal Classification (CTC). Despite significant innovation on these benchmarks, all leading published prior work relies on a WFST-based CTC decoder that requires ${\sim}$320 GB of RAM. These memory requirements limit accessibility for both patients and researchers. Here, we propose LightBeam, a non-WFST based CTC decoder that requires only ${\sim}$10 GB of RAM and achieves state-of-the-art performance on both benchmarks. LightBeam achieves this by integrating an LLM into the beam-search process via delayed fusion, obviating the prior need for using a large N-gram LM. LightBeam is implemented in Python and is open-source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。