arXiv:2410.15319cs.CLcs.AI2024-10被引 28

让大模型从学说话转向懂因果,提升可信度与公平性

Causality for Large Language Models

  • 将因果机制融入大模型训练全流程,突破纯统计关联
  • 揭示当前大模型只是'因果鹦鹉',会复述但不懂应用
  • 适合关注AI伦理、可解释性与鲁棒性的研究者

大语言模型(LLMs)虽在语言任务中取得突破,但仍依赖概率建模,易捕捉语言模式和社交偏见中的虚假相关,而非真实因果关系。这导致模型存在性别、种族等偏差及幻觉问题。现有研究多通过提示工程激活因果知识或建立评估基准,但依赖人工干预,无法实现深层因果理解。本文提出应将因果性嵌入模型生命周期的每个阶段——从词嵌入学习、基础模型训练、微调、对齐、推理到评估,以构建更可解释、可靠且具备因果认知的智能系统。同时,本文梳理了六个前沿方向,推动大模型向更通用、更智能的因果推理演进。

原文摘要 · Abstract (English)

Recent breakthroughs in artificial intelligence have driven a paradigm shift, where large language models (LLMs) with billions or trillions of parameters are trained on vast datasets, achieving unprecedented success across a series of language tasks. However, despite these successes, LLMs still rely on probabilistic modeling, which often captures spurious correlations rooted in linguistic patterns and social stereotypes, rather than the true causal relationships between entities and events. This limitation renders LLMs vulnerable to issues such as demographic biases, social stereotypes, and LLM hallucinations. These challenges highlight the urgent need to integrate causality into LLMs, moving beyond correlation-driven paradigms to build more reliable and ethically aligned AI systems. While many existing surveys and studies focus on utilizing prompt engineering to activate LLMs for causal knowledge or developing benchmarks to assess their causal reasoning abilities, most of these efforts rely on human intervention to activate pre-trained models. How to embed causality into the training process of LLMs and build more general and intelligent models remains unexplored. Recent research highlights that LLMs function as causal parrots, capable of reciting causal knowledge without truly understanding or applying it. These prompt-based methods are still limited to human interventional improvements. This survey aims to address this gap by exploring how causality can enhance LLMs at every stage of their lifecycle-from token embedding learning and foundation model training to fine-tuning, alignment, inference, and evaluation-paving the way for more interpretable, reliable, and causally-informed models. Additionally, we further outline six promising future directions to advance LLM development, enhance their causal reasoning capabilities, and address the current limitations these models face.

大模型因果推理AI伦理可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。