用微调模型自动识别数据-文本对中的错误，提供可解释的评分。
XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signals
- 微调语言模型判断数据单元的遗漏、多余、错误或正确
- 在多个基准上优于大模型判别方法，且与人工评分高度相关
- 输出诊断反馈，可用于数据-文本生成的优化改进
评估数据-文本对齐仍具挑战：现有度量难以提供评分依据，而基于提示的大模型判别方法成本高且不可靠。本文提出一种端到端可解释的评估度量，通过微调语言模型识别数据-文本配对中遗漏、多余、错误和正确的数据单元。这些局部判断被聚合为精确率、召回率和F1分数，既提供细粒度诊断反馈，也给出可解释的对齐质量度量。在多个基准上，微调模型在错误预测方面优于大模型判别方法，并取得具有竞争力的精确率、召回率和F1分数，同时与人工评判保持强相关性。除评估外，验证器输出还为下游修正和优化提供了有用反馈信号，支持数据到文本及文本到数据任务的对齐改进。代码与资源见 https://github.com/guihuzhang/xqdt。
原文摘要 · Abstract (English)
Evaluating data-text alignment remains challenging: existing metrics often provide limited explanations for the scores, while prompt-based LLM-as-Judge methods can be expensive and unreliable. We present an end-to-end explainable evaluation metric that fine-tunes a language model to identify omitted, extra, incorrect, and correct data units in a data-text pair. These local judgements are aggregated into precision, recall, and F1 scores, providing both fine-grained diagnostic feedback and an interpretable measure of alignment quality. Across benchmarks, our fine-tuned models outperform LLM-as-Judge methods in error prediction and achieve competitive precision, recall, and F1 scores, while maintaining strong correlation with human judgements. Beyond evaluation, our verifier outputs also provide useful feedback signals for downstream correction and refinement, supporting alignment-oriented improvement of data-to-text and text-to-data. Code and resources are available at https://github.com/guihuzhang/xqdt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。