比较经典AI与大模型在医保决策中的对齐效果,发现两者表现接近。
Classical AI vs. LLMs for Decision-Maker Alignment in Health Insurance Choices
- 用经典推理与大模型提示工程对比决策对齐
- 两类方法在不同风险偏好下对齐率相近,中等风险时经典方法略优
- 适合关注医疗决策、人机对齐的研究者参考
随着算法决策在高风险领域应用增多,AI对齐研究从通用价值对齐转向考虑决策者属性的特定情境方法。已有研究提出两种主要策略:(1) 结合案例推理、贝叶斯推理和自然决策的经典AI方法;(2) 借助提示工程的大语言模型(LLM)方法。尽管两者在医疗分诊等有限场景中表现良好,其在新情境下的泛化能力仍待验证。本文实现一个经典AI模型,并开发基于大模型的决策系统,分别使用大推理模型(GPT-5)和非推理模型(GPT-4),在零样本提示框架下结合加权自洽性进行评估。在标注了三种风险容忍度(0.0, 0.5, 1.0)的健康保险决策数据集上,经典AI与LLM方法在属性目标对齐方面表现相当,经典方法在中等风险偏好下略有优势。数据集与开源代码已公开于:https://github.com/TeX-Base/ClassicalAIvsLLMsforDMAlignment 及 https://github.com/Parallax-Advanced-Research/ITM/tree/feature_insurance。
原文摘要 · Abstract (English)
As algorithmic decision-makers are increasingly applied to high-stakes domains, AI alignment research has evolved from a focus on universal value alignment to context-specific approaches that account for decision-maker attributes. Prior work on Decision-Maker Alignment (DMA) has explored two primary strategies: (1) classical AI methods integrating case-based reasoning, Bayesian reasoning, and naturalistic decision-making, and (2) large language model (LLM)-based methods leveraging prompt engineering. While both approaches have shown promise in limited domains such as medical triage, their generalizability to novel contexts remains underexplored. In this work, we implement a prior classical AI model and develop an LLM-based algorithmic decision-maker evaluated using a large reasoning model (GPT-5) and a non-reasoning model (GPT-4) with weighted self-consistency under a zero-shot prompting framework, as proposed in recent literature. We evaluate both approaches on a health insurance decision-making dataset annotated for three target decision-makers with varying levels of risk tolerance (0.0, 0.5, 1.0). In the experiments reported herein, classical AI and LLM-based models achieved comparable alignment with attribute-based targets, with classical AI exhibiting slightly better alignment for a moderate risk profile. The dataset and open-source implementation are publicly available at: https://github.com/TeX-Base/ClassicalAIvsLLMsforDMAlignment and https://github.com/Parallax-Advanced-Research/ITM/tree/feature_insurance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。