用稀疏注意力机制减少大模型推理成本，提升思维链效率。
Reducing Reasoning Costs: The Path of Optimization for Chain of Thought via Sparse Attention Mechanism
- 采用仅关注关键词元的稀疏注意力机制优化思维链。
- 在MIT线性代数题上，推理时间与思维链长度显著低于o1 Preview。
- 适合关注大模型推理加速与高效思维链设计的研究者。
为应对大语言模型中思维链导致的推理成本激增问题，本研究提出一种仅聚焦少数相关词元的稀疏注意力机制。研究人员构建了新型注意力机制，并以自定义GPT训练的GiantRabbit作为实验工具，对比其与o1 Preview在解决MIT OpenCourseWare线性代数测试题时的推理时间、正确率及思维链长度。结果表明，GiantRabbit在推理时间与思维链长度上均显著优于o1 Preview，验证了稀疏注意力机制在优化思维链推理中的可行性。详细架构设计与实验流程已上传至GitHub：https://github.com/brucewang123456789/GeniusTrail.git。
原文摘要 · Abstract (English)
In order to address the chain of thought in the large language model inference cost surge, this research proposes to use a sparse attention mechanism that only focuses on a few relevant tokens. The researcher constructed a new attention mechanism and used GiantRabbit trained with custom GPTs as an experimental tool. The experiment tested and compared the reasoning time, correctness score and chain of thought length of this model and o1 Preview in solving the linear algebra test questions of MIT OpenCourseWare. The results show that GiantRabbit's reasoning time and chain of thought length are significantly lower than o1 Preview. It verifies the feasibility of sparse attention mechanism for optimizing chain of thought reasoning. Detailed architectural details and experimental process have been uploaded to Github, the link is:https://github.com/brucewang123456789/GeniusTrail.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。