量化压缩下大模型因果推理仍稳定,图增强可缓解干预推理退化。
Compressed Causal Reasoning: Quantization and GraphRAG Effects on Interventional and Counterfactual Accuracy
- 用分层基准测试量化对因果三层次推理的影响,覆盖关联、干预、反事实。
- NF4量化使整体准确率下降不足1%,干预推理最敏感,反事实存在类型差异性弱点。
- 引入真实因果图的GraphRAG提升干预准确率1.7%,现有反事实数据集难暴露深层缺陷。
大语言模型在关联、干预和反事实推理中的因果推理对高风险决策至关重要。随着部署向边缘和资源受限环境转移,INT8和NF4等量化模型已成为标准。然而,精度降低对形式化因果推理的影响尚不明确。本研究首次系统评估了量化在珍珠因果阶梯三个层级上的影响。基于3000样本分层的CLadder基准测试发现,Llama 3 8B在量化下各层级准确率总体保持稳定,NF4仅导致不足1%的整体性能下降。第二层级干预查询对精度损失最敏感,第三层级反事实推理相对稳定但不同查询类型表现不均,如碰撞器偏差和后门调整存在异质性弱点。在CRASS基准上,不同精度间性能近乎一致,表明现有常识反事实数据集缺乏揭示量化引发推理漂移的结构敏感性。进一步引入基于真实因果图的GraphRAG,观察到NF4干预准确率提升1.7%,部分抵消压缩带来的退化。结果表明,因果推理对四比特量化具有意外鲁棒性,图结构增强可选择性强化干预推理,而当前反事实基准未能捕捉深层因果脆弱性。本工作为压缩因果推理提供了初步实证地图,并为高效且结构支持的因果AI部署提供实践指导。
原文摘要 · Abstract (English)
Causal reasoning in Large Language Models spanning association, intervention, and counterfactual inference is essential for reliable decision making in high stakes settings. As deployment shifts toward edge and resource constrained environments, quantized models such as INT8 and NF4 are becoming standard. Yet the impact of precision reduction on formal causal reasoning is poorly understood. To our knowledge, this is the first study to systematically evaluate quantization effects across all three levels of Pearls Causal Ladder. Using a 3000 sample stratified CLadder benchmark, we find that rung level accuracy in Llama 3 8B remains broadly stable under quantization, with NF4 showing less than one percent overall degradation. Interventional queries at rung 2 are the most sensitive to precision loss, whereas counterfactual reasoning at rung 3 is comparatively stable but exhibits heterogeneous weaknesses across query types such as collider bias and backdoor adjustment. Experiments on the CRASS benchmark show near identical performance across precisions, indicating that existing commonsense counterfactual datasets lack the structural sensitivity needed to reveal quantization induced reasoning drift. We further evaluate Graph Retrieval Augmented Generation using ground truth causal graphs and observe a consistent improvement in NF4 interventional accuracy of plus 1.7 percent, partially offsetting compression related degradation. These results suggest that causal reasoning is unexpectedly robust to four bit quantization, graph structured augmentation can selectively reinforce interventional reasoning, and current counterfactual benchmarks fail to capture deeper causal brittleness. This work provides an initial empirical map of compressed causal reasoning and practical guidance for deploying efficient and structurally supported causal AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。