让大模型生成专家级解释，自动评估其专业思维一致性。
T-FIX: Text-Based Explanations with Features Interpretable to eXperts
- 用专家定义的领域标准，评估大模型解释是否符合专业推理逻辑。
- 在三个领域七项任务中验证，无需持续人工标注即可评估新解释。
- 适合医疗、天文等高专业度场景，支持个性化评估需求。
随着大语言模型在知识密集型场景（如外科手术、天文学、心理治疗）中的应用，用户常为领域专家，他们不仅需要答案，还需符合专业思维的解释。然而，评估大模型是否‘像专家一样思考’仍具挑战：现有方法依赖逐例专家标注，成本高、难扩展，且受限于单一正确推理标准。为此，我们提出 T-FIX，一个统一评估框架，将专家对齐作为大模型解释的核心属性。T-FIX覆盖三个领域共七项科学任务，每项任务基于专家定义的标准评估领域化推理能力，而非通用解释质量。该框架实现无需持续专家参与的自动化、可定制化评估，可泛化至未见解释。代码已公开于 https://github.com/BrachioLab/FIX-2/。
原文摘要 · Abstract (English)
As LLMs are deployed in knowledge-intensive settings (e.g., surgery, astronomy, therapy), users are often domain experts who expect not just answers, but explanations that mirror professional reasoning. Yet evaluating whether an LLM "thinks like an expert" remains difficult: existing approaches rely on per-example expert annotation, making them costly, hard to scale, and tied to a single notion of correct reasoning within each domain. To address this gap, we introduce T-FIX, a unified evaluation framework that operationalizes expert alignment as a desired attribute of LLM-generated explanations. T-FIX spans seven scientific tasks across three domains, with each task evaluated against expert-defined criteria that capture domain-grounded reasoning rather than generic explanation quality. Our framework enables automatic, personalizable evaluation of expert alignment that generalizes to unseen explanations without ongoing expert involvement. Code is available at https://github.com/BrachioLab/FIX-2/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。