用小模型提升数学语音转公式准确率，让讲题更清晰。
MathSpeech: Leveraging Small LMs for Accurate Conversion in Mathematical Speech-to-Formula
- 结合语音识别与120M小模型，自动纠错并转为LaTeX格式。
- 在字符错误率上降至0.298，优于GPT-4o。
- 适合数学教学、学术演讲等需精准转录场景。
在数学讲座或科研报告等场景中，口头表达数学公式常因缺乏视觉辅助而影响理解，尤其对听障者或语言障碍者更为困难。当前语音识别系统常输出冗长且错误的文本描述（如“e to the power of i x equals cosine of x plus i side of x”），而非标准的LaTeX格式（如$ e^{ix} = an(x) $）。为此，本文提出MathSpeech，一种融合ASR与小语言模型（sLM）的新型转换管道，能将口语化数学表达准确转为结构化LaTeX。在基于讲座录音构建的新数据集上评估显示，该方法使用仅120M参数的小模型，其LaTeX生成性能接近领先商用大模型（LLMs）。在字符错误率（CER）、BLEU和ROUGE指标上，相较于GPT-4o，CER从0.390降至0.298，且各项得分更高。
原文摘要 · Abstract (English)
In various academic and professional settings, such as mathematics lectures or research presentations, it is often necessary to convey mathematical expressions orally. However, reading mathematical expressions aloud without accompanying visuals can significantly hinder comprehension, especially for those who are hearing-impaired or rely on subtitles due to language barriers. For instance, when a presenter reads Euler's Formula, current Automatic Speech Recognition (ASR) models often produce a verbose and error-prone textual description (e.g., e to the power of i x equals cosine of x plus i $\textit{side}$ of x), instead of the concise $\LaTeX{}$ format (i.e., $ e^{ix} = \cos(x) + i\sin(x) $), which hampers clear understanding and communication. To address this issue, we introduce MathSpeech, a novel pipeline that integrates ASR models with small Language Models (sLMs) to correct errors in mathematical expressions and accurately convert spoken expressions into structured $\LaTeX{}$ representations. Evaluated on a new dataset derived from lecture recordings, MathSpeech demonstrates $\LaTeX{}$ generation capabilities comparable to leading commercial Large Language Models (LLMs), while leveraging fine-tuned small language models of only 120M parameters. Specifically, in terms of CER, BLEU, and ROUGE scores for $\LaTeX{}$ translation, MathSpeech demonstrated significantly superior capabilities compared to GPT-4o. We observed a decrease in CER from 0.390 to 0.298, and higher ROUGE/BLEU scores compared to GPT-4o.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。