测试三种策略提升数值事实核查效果，发现证据质量比上下文长度更重要。
DS@GT at CheckThat! 2025: Evaluating Context and Tokenization Strategies for Numerical Fact Verification
- 用ModernBERT测试更长上下文和右到左分词对数值判断的影响。
- 最长上下文与右到左分词均未提升效果，最高F1为0.57。
- 适合关注数值类事实核查的NLP研究者，代码已开源。
数值类陈述（涉及数量、比较和时间参考）给自动化事实核查系统带来独特挑战。本研究基于QuanTemp数据集，构建自有的证据检索管道，评估三种建模策略：(1) 使用ModernBERT时增加证据数量并扩大输入上下文窗口；(2) 采用右到左（R2L）分词方式；(3) 两者联合影响。与算术推理任务中结果相反，R2L分词并未提升数值任务中的自然语言推理表现。更长的上下文窗口也未能提高分类性能，凸显证据质量是主要瓶颈。我们最佳系统在CheckThat! 2025任务3中取得0.57的宏平均F1，位列前四。代码已公开于https://github.com/dsgt-arc/checkthat-2025-numerical。
原文摘要 · Abstract (English)
Numerical claims, statements involving quantities, comparisons, and temporal references, pose unique challenges for automated fact-checking systems. In this study, we evaluate modeling strategies for veracity prediction of such claims using the QuanTemp dataset and building our own evidence retrieval pipeline. We investigate three key factors: (1) the impact of more evidences with longer input context windows using ModernBERT, (2) the effect of right-to-left (R2L) tokenization, and (3) their combined influence on classification performance. Contrary to prior findings in arithmetic reasoning tasks, R2L tokenization does not boost natural language inference (NLI) of numerical tasks. A longer context window does also not enhance veracity performance either, highlighting evidence quality as the dominant bottleneck. Our best-performing system achieves competitive macro-average F1 score of 0.57 and places us among the Top-4 submissions in Task 3 of CheckThat! 2025. Our code is available at https://github.com/dsgt-arc/checkthat-2025-numerical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。