用大模型当裁判评估时间序列解释，发现判对比生成更可靠。
TSQueryBench: LLM-as-a-Judge for Time Series Explanations
- 设计合成数据集，用大模型判断时间序列解释的对错。
- 6个大模型在4项任务中表现不一，生成错误时仍能准确识别正确解释。
- 适合做数值型推理评估的自动化工具，尤其适用于高风险领域。
基础模型在高风险领域生成时间序列的自然语言解释日益普遍，其事实正确性至关重要。评估这类解释与标准文本生成不同：正确性需基于结构化数据验证数值陈述，而非文本相似度。尽管大模型作为裁判已成可扩展的评估范式，但其在数值驱动的时间序列解释评估中的适用性尚无研究。我们提出TSQueryBench，一个包含500个时间序列实例的可控合成基准，覆盖10种查询类型，每例配以正确、部分正确和错误的解释。我们在四项任务中评估了六个大语言模型：解释生成、相对排序、独立评分及多异常检测。核心发现是生成评估存在系统性不对称：即使模型无法生成数值正确的解释，仍能可靠识别或评分正确的解释。结果表明，基于规则引导的大模型评估在时间序列推理中比生成更可靠，支持大模型裁判在数值驱动场景下的规模化应用。代码与数据：https://github.com/Prxxthxm/TSQueryBench/
原文摘要 · Abstract (English)
Natural language explanations of time series data are increasingly produced by foundation models in high stakes domains, making factual correctness critical. Evaluating such explanations differs fundamentally from standard natural language generation: correctness requires verifying numerical claims against structured data rather than similarity to reference text. While LLM as a Judge has emerged as a scalable paradigm for text evaluation, its applicability to numerically grounded time series explanations remains unstudied. We introduce TSQueryBench, a controlled synthetic benchmark of 500 time series instances across 10 query types, each paired with correct, partially correct, and incorrect explanations. We evaluate six large language models across four tasks: explanation generation, relative ranking, independent scoring, and multi anomaly detection. Our central finding is a consistent generation evaluation asymmetry: models that fail to generate numerically correct explanations nonetheless reliably identify or score correct ones. These results show that rubric guided LLM evaluation is substantially more reliable than generation for time series reasoning, supporting LLM judges as scalable evaluators in numerically grounded settings. Code and data: https://github.com/Prxxthxm/TSQueryBench/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。