让大模型在推理时更准确理解上下文,提升计算类问题得分。
SIFT: Grounding LLM Reasoning in Contexts via Stickers
- 用模型自动生成关键信息贴纸,强化上下文理解
- 使DeepSeek-R1在AIME2024上准确率从78.33%提升至85.67%
- 适用于需精准逻辑推理的数学与复杂问答场景
本文指出大语言模型在推理过程中常误解上下文,从小型模型Llama3.2-3B-Instruct到前沿模型DeepSeek-R1均存在此问题。例如,“每公斤10美元”中的“per”可能被误读,导致计算错误。为此提出一种新的后训练方法SIFT,通过增加推理时计算量,将模型推理过程锚定在上下文中。SIFT的核心是‘贴纸’(Sticker),由模型自动生成,用于显式强调上下文关键信息。基于筛选后的贴纸,SIFT生成两个预测:原始查询结果与贴纸增强查询结果。若二者不同,则通过正向优化(使提取事实更契合问题)和逆向生成(符合模型内在倾向)对贴纸进行迭代优化,以获得更可信的推理结果。在多种模型(3B至100B+)和基准测试(如GSM8K、MATH-500)中均实现稳定提升。尤其在AIME2024上,SIFT将DeepSeek-R1的pass@1准确率从78.33%提升至85.67%,创下开源社区新纪录。代码已公开于https://github.com/zhijie-group/SIFT。
原文摘要 · Abstract (English)
This paper identifies the misinterpretation of the context can be a significant issue during the reasoning process of large language models, spanning from smaller models like Llama3.2-3B-Instruct to cutting-edge ones like DeepSeek-R1. For example, in the phrase "10 dollars per kilo," LLMs might not recognize that "per" means "for each," leading to calculation errors. We introduce a novel, post-training approach called **Stick to the Facts (SIFT)** to tackle this. SIFT leverages increasing inference-time compute to ground LLM reasoning in contexts. At the core of SIFT lies the *Sticker*, which is generated by the model itself to explicitly emphasize the key information within the context. Given the curated Sticker, SIFT generates two predictions -- one from the original query and one from the query augmented with the Sticker. If they differ, the Sticker is sequentially refined via *forward* optimization (to better align the extracted facts with the query) and *inverse* generation (to conform with the model's inherent tendencies) for more faithful reasoning outcomes. Studies across diverse models (from 3B to 100B+) and benchmarks (e.g., GSM8K, MATH-500) reveal consistent performance improvements. Notably, SIFT improves the pass@1 accuracy of DeepSeek-R1 on AIME2024 from 78.33% to **85.67**%, establishing a new state-of-the-art in the open-source community. The code is available at https://github.com/zhijie-group/SIFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。