让小模型学会大模型的因果推理逻辑，生成有条理的解释。
Causal Distillation: Transferring Structured Explanations from Large to Compact Language Models
- 用大模型的因果解释指导小模型学习，生成结构一致的因果链。
- 提出新评估指标CEC，量化解释的逻辑连贯性与覆盖度。
- 适合需要可解释推理的轻量级AI应用，如医疗、金融决策支持。
大型专有语言模型展现出强大的因果推理能力，而小型开源模型难以复现。我们提出一种新型框架，将大模型的因果解释能力蒸馏到紧凑的开源模型中。核心思想是训练小模型生成与教师模型一致的结构化因果解释。为评估学生模型生成解释的质量，引入新指标因果解释一致性（CEC），通过句级语义对齐衡量生成解释与教师参考之间的对应关系，捕捉因果链的忠实性与覆盖度。该框架与CEC指标为小模型实现稳健因果推理提供了系统性训练基础，并可规范化评估语言模型输出解释的连贯性。
原文摘要 · Abstract (English)
Large proprietary language models exhibit strong causal reasoning abilities that smaller open-source models struggle to replicate. We introduce a novel framework for distilling causal explanations that transfers causal reasoning skills from a powerful teacher model to a compact open-source model. The key idea is to train the smaller model to develop causal reasoning abilities by generating structured cause-and-effect explanations consistent with those of the teacher model. To evaluate the quality of the student-generated explanations, we introduce a new metric called Causal Explanation Coherence (CEC) to assess the structural and logical consistency of causal reasoning. This metric uses sentence-level semantic alignment to measure how well each part of the generated explanation corresponds to the teacher's reference, capturing both faithfulness and coverage of the underlying causal chain. Our framework and the CEC metric provide a principled foundation for training smaller models to perform robust causal reasoning and for systematically assessing the coherence of explanations in language model outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。