研究存内计算芯片上大模型推理能力的退化与应对策略
Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
- 分析存阻器件非理想性对大模型推理的影响机制
- 发现浅层模块冗余最有效，思维模式在低噪声下表现好
- 适合关注边缘AI部署可靠性的研究人员
基于存阻的模拟存内计算（CIM）架构因能效和计算密度优势，成为高效部署大语言模型（LLMs）的潜在方案。然而，存阻器件固有的非理想性导致精度下降。本文首次系统研究此类非理想性对LLM推理能力的影响，实证发现推理性能显著下降且不同基准测试结果差异明显。进一步评估了三种无需训练的鲁棒性增强策略：思维模式、上下文学习和模块冗余。研究总结出有效指导原则：浅层冗余对提升鲁棒性尤为有效；思维模式在低噪声下表现更优但高噪声时退化；上下文学习虽略微牺牲性能但可缩短输出长度。研究成果为非理想条件下大模型推理提供了新洞见与实用策略。
原文摘要 · Abstract (English)
Memristor-based analog compute-in-memory (CIM) architectures provide a promising substrate for the efficient deployment of Large Language Models (LLMs), owing to superior energy efficiency and computational density. However, these architectures suffer from precision issues caused by intrinsic non-idealities of memristors. In this paper, we first conduct a comprehensive investigation into the impact of such typical non-idealities on LLM reasoning. Empirical results indicate that reasoning capability decreases significantly but varies for distinct benchmarks. Subsequently, we systematically appraise three training-free strategies, including thinking mode, in-context learning, and module redundancy. We thus summarize valuable guidelines, i.e., shallow layer redundancy is particularly effective for improving robustness, thinking mode performs better under low noise levels but degrades at higher noise, and in-context learning reduces output length with a slight performance trade-off. Our findings offer new insights into LLM reasoning under non-ideality and practical strategies to improve robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。