构建首个多语言语音对话事实核查数据集,支持三语真实对话验证。
TRILOGUE: A Trilingual Spoken Dialogue Fact-Checking Benchmark with Evidence and Paired Audio

- 构建跨语言语音对话数据集,含三语转录与音频对齐
- 近12,000段对话中,俄语和哈萨克语近5,000段为真人录音
- 首次支持端到端语音对话事实核查评估,尤其适合多语种研究者
现代虚假信息常以语音形式传播,但现有事实核查系统仍主要基于书面文本评估。语音对话中的主张常分散于不同发言者与回合,依赖上下文,且自动语音识别(ASR)错误会进一步干扰验证。现有资源规模小、以英语为主或仅关注标注而非端到端评估,缺乏大规模多语言带音频的基准。本文提出TRILOGUE(三语语音对话事实核查基准),包含英语、俄语、哈萨克语的近12,000段源基语音对话,共187,000个回合,390小时配对音频与转录文本,含约5,000段真人录制的俄语与哈萨克语对话,具备逐词时间戳对齐。该数据集支持主张可核查性检测、源文章证据检索及三种输入下的主张验证:仅主张、金标准证据、检索证据。基线结果显示,ASR降质与跨语言迁移仍是挑战,尤其在哈萨克语上,而检索来源证据显著缩小了与金标准验证的差距。
原文摘要 · Abstract (English)
Modern misinformation is often heard before it is read, yet fact-checking systems are still evaluated mainly on clean written claims. Spoken dialogue remains different even when systems operate on transcripts: claims may be distributed across speakers and turns, depend on prior context, and become harder to verify when Automatic Speech Recognition (ASR) errors distort the available text. Prior spoken dialogue fact-checking resources are small, English-centric, or focused on annotation rather than end-to-end benchmarking, leaving no large multilingual benchmark with paired speech and turn-level labels. We introduce TRILOGUE (TRIlingual spoken diaLOGUE fact-checking), a large-scale trilingual benchmark of source-grounded spoken dialogues in English, Russian, and Kazakh. It contains nearly 12K dialogues, 187K turns, and 390 hours of paired audio with ASR transcripts and word-level timestamp alignments across all three languages, including nearly 5K human-recorded Russian and Kazakh dialogue files. TRILOGUE supports claim check-worthiness detection, source-article evidence retrieval, and claim verification with claim-only, gold-evidence, and retrieved-evidence inputs. Baselines show that ASR degradation and cross-lingual transfer remain challenging, especially for Kazakh, while retrieved source evidence substantially narrows the gap to gold-evidence verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。