Pisets提升讲座与访谈语音转录准确率，减少幻觉错误。
Pisets: A Robust Speech Recognition System for Lectures and Interviews
- 三阶段架构：Wav2Vec2主识别+AST滤除误报+Whisper精修
- 在俄语语音数据上表现优于WhisperX和原版Whisper，长音频更稳定
- 适合科研人员与记者使用，开源代码已发布
本文提出面向科学家与记者的语音转文字系统Pisets，采用三组件架构以提升语音识别准确率并降低Whisper模型的错误与幻觉。该架构包含基于Wav2Vec2的主识别、通过音频频谱变换器（AST）进行假阳性过滤，以及利用Whisper完成最终转录。结合课程学习方法与多样化的俄语语音语料库，显著提升了系统性能。此外，引入先进的不确定性建模技术，进一步优化了转录质量。所提方法在多种声学条件下对长时音频的转录表现出更强鲁棒性，优于WhisperX和标准Whisper模型。Pisets系统源代码已在GitHub公开：https://github.com/bond005/pisets。
原文摘要 · Abstract (English)
This work presents a speech-to-text system "Pisets" for scientists and journalists which is based on a three-component architecture aimed at improving speech recognition accuracy while minimizing errors and hallucinations associated with the Whisper model. The architecture comprises primary recognition using Wav2Vec2, false positive filtering via the Audio Spectrogram Transformer (AST), and final speech recognition through Whisper. The implementation of curriculum learning methods and the utilization of diverse Russian-language speech corpora significantly enhanced the system's effectiveness. Additionally, advanced uncertainty modeling techniques were introduced, contributing to further improvements in transcription quality. The proposed approaches ensure robust transcribing of long audio data across various acoustic conditions compared to WhisperX and the usual Whisper model. The source code of "Pisets" system is publicly available at GitHub: https://github.com/bond005/pisets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。