用大模型提升长篇中文语音转写文本的纠错能力。
Full-text Error Correction for Chinese Speech Recognition with Large Language Model
- 构建中文长文本纠错数据集ChFT,支持上下文级纠错。
- 微调大模型在长文本上表现良好,不同提示策略各有优势。
- 适合语音转写后处理、会议/播客等长音频场景使用。
大语言模型在自动语音识别(ASR)纠错中展现出巨大潜力,但现有研究多集中于短时语音片段,而忽略长篇录音(如播客、新闻、会议)生成的全文转写文本。本文提出针对此类长文本的纠错任务,构建了名为ChFT的中文全文字纠错数据集,通过文本到语音合成、ASR生成及错误对提取的流水线实现。该数据集支持跨上下文纠错,涵盖标点恢复、逆文本规范化等多种错误类型,使纠错更全面。进一步地,采用多样化提示与输出格式对预训练大模型进行微调,并在同质、最新及困难测试集上评估性能。实验表明,微调后的模型在不同提示策略下均表现良好,为后续研究建立了可靠基准。数据集已公开。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated substantial potential for error correction in Automatic Speech Recognition (ASR). However, most research focuses on utterances from short-duration speech recordings, which are the predominant form of speech data for supervised ASR training. This paper investigates the effectiveness of LLMs for error correction in full-text generated by ASR systems from longer speech recordings, such as transcripts from podcasts, news broadcasts, and meetings. First, we develop a Chinese dataset for full-text error correction, named ChFT, utilizing a pipeline that involves text-to-speech synthesis, ASR, and error-correction pair extractor. This dataset enables us to correct errors across contexts, including both full-text and segment, and to address a broader range of error types, such as punctuation restoration and inverse text normalization, thus making the correction process comprehensive. Second, we fine-tune a pre-trained LLM on the constructed dataset using a diverse set of prompts and target formats, and evaluate its performance on full-text error correction. Specifically, we design prompts based on full-text and segment, considering various output formats, such as directly corrected text and JSON-based error-correction pairs. Through various test settings, including homogeneous, up-to-date, and hard test sets, we find that the fine-tuned LLMs perform well in the full-text setting with different prompts, each presenting its own strengths and weaknesses. This establishes a promising baseline for further research. The dataset is available on the website.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。