用简单方法提升语音转录中模糊引用的精准定位能力
TimeStampEval: A Simple LLM Eval and a Little Fuzzy Matching Trick to Improve Search Accuracy
- 先用快速匹配预筛选，再让大模型验证短片段，提升效率
- 在长转录文本中定位模糊引用，准确率最高达90%以上
- 适合需要高精度时间戳检索的播客自动化与记录对齐场景
传统模糊匹配在处理语义相同但句式不同的引文时表现不佳，尤其在官方记录与语音转录间对齐时。我们提出TimeStampEval基准，用于从长篇转录中检索精确毫秒级时间戳。采用两阶段方法，在降低推理成本超90%的同时显著提升准确率。实验显示：（1）提示设计比模型选择更重要，优化后准确率提升3-20点，令牌数减少30-40%；（2）错位误差为独立类别，表明模型理解任务但边界判断不准；（3）600-850令牌的轻量推理预算使弱模型准确率从37%升至77%，强模型突破90%；（4）‘辅助模糊匹配’策略——使用RapidFuzz预过滤+大模型在短片段验证——将模糊匹配准确率提升最多50点，延迟减半，每正确结果成本降低达96%。在10个不同长度（50k-900k tokens）、跨度1989-2025年的转录上测试，对目标不存在情况的拒绝准确率保持95%-100%。
原文摘要 · Abstract (English)
Traditional fuzzy matching often fails when searching for quotes that are semantically identical but syntactically different across documents-a common issue when aligning official written records with speech-to-text transcripts. We introduce TimeStampEval, a benchmark for retrieving precise millisecond timestamps from long transcripts given non-verbatim quotes. Our simple two-stage method dramatically improves retrieval accuracy while cutting inference costs by over 90%. The motivating use case is an automated long-form podcast that assembles Congressional Record clips into AI-hosted narration. The technical challenge: given a sentence-timestamped transcript and a target quote that may differ due to transcription or editorial drift, return exact start and end boundaries. Standard algorithms handle verbatim text but break under fuzzier variants. Evaluating six modern LLMs on a 2,800-sentence (120k-token) transcript revealed four key findings. (1) Prompt design matters more than model choice: placing the query before the transcript and using compact formatting improved accuracy by 3-20 points while reducing token count by 30-40%. (2) Off-by-one errors form a distinct category, showing models understand the task but misplace boundaries. (3) A modest reasoning budget (600-850 tokens) raises accuracy from 37% to 77% for weak setups and to above 90% for strong ones. (4) Our "Assisted Fuzzy" approach-RapidFuzz pre-filtering followed by LLM verification on short snippets-improves fuzzy match accuracy by up to 50 points while halving latency and reducing cost per correct result by up to 96%. Extended tests on ten transcripts (50k-900k tokens, 1989-2025) confirm robustness to transcript length, vocabulary drift, and domain change, maintaining 95-100% rejection accuracy for absent targets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。