arXiv:2601.18732econ.THcs.AI2026-01被引 2

分离训练与决策可提升AI系统灵活性,适配不同目标。

Optimal Use of Preferences in Artificial Intelligence Algorithms

  • 先训练无偏好模型,再在决策阶段加偏好,更通用
  • 偏好嵌入会降低信息价值,导致后验概率压缩
  • 人机协作时嵌入偏好更优,因能省去复杂判断

机器学习系统通过训练损失或后处理校准预测来嵌入偏好。本文基于Strack和Yang(2024)的信息设计方法,给出一种不依赖具体决策问题的最优条件:将偏好分离于训练阶段,在决策阶段事后应用为最优策略。相比以往需预设下游目标的方法,该福利结果适用于所有决策场景。核心前提是‘信息边际价值递减’:相对于固定(归一化)的无偏好损失,嵌入偏好会削弱信息的边际价值,导致学习后验分布发生均值不变收缩。由于信息价值在信念上呈凸性,无偏好训练对任意期望效用决策问题均弱占优。这为模块化AI流水线提供了理论基础——先学习校准概率,再通过下游决策规则实现非对称成本。但分离要求用户实现最优决策规则;当认知约束存在(如人类在人机决策中所见),偏好嵌入反而更优,因其自动完成阈值计算。因此设计建议是:若目标可能变化,保留后处理选项;若决策阶段存在认知摩擦,则应嵌入偏好。

原文摘要 · Abstract (English)

Machine learning systems embed preferences either in training losses or through post-processing of calibrated predictions. Applying information design methods from Strack and Yang (2024), this paper provides decision problem agnostic conditions under which separation training preference free and applying preferences ex post is optimal. Unlike prior work that requires specifying downstream objectives, the welfare results here apply uniformly across decision problems. The key primitive is a diminishing-value-of-information condition: relative to a fixed (normalised) preference-free loss, preference embedding makes informativeness less valuable at the margin, inducing a mean-preserving contraction of learned posteriors. Because the value of information is convex in beliefs, preference-free training weakly dominates for any expected utility decision problem. This provides theoretical foundations for modular AI pipelines that learn calibrated probabilities and implement asymmetric costs through downstream decision rules. However, separation requires users to implement optimal decision rules. When cognitive constraints bind, as documented in human AI decision-making, preference embedding can dominate by automating threshold computation. These results provide design guidance: preserve optionality through post-processing when objectives may shift; embed preferences when decision-stage frictions dominate.

AI设计偏好建模决策优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。