兼顾生成成本与人觉差异,提升偏好优化效率
Consecutive Preferential Bayesian Optimization
- 约束比较对象为已有候选,降低生成成本
- 引入可察觉差异阈值,处理人类反馈模糊性
- 适合高成本生成或主观反馈场景
偏好贝叶斯优化适用于难以直接测量或成本高昂的目标优化,仅依赖少量人工对比判断。现有方法忽略候选生成成本,本文提出连续偏好贝叶斯优化(Consecutive Preferential Bayesian Optimization),通过限制比较仅在已生成候选间进行,显著降低生产成本。同时,引入可察觉差异(Just-Noticeable Difference)阈值,建模人类对微小效用差异的感知模糊性。采用信息论采集策略,在考虑感知模糊性的偏好模型下,选择最有助于定位最优解的新配置。实验表明,在高生产成本或存在反馈惰性条件下,该方法准确率有明显提升。
原文摘要 · Abstract (English)
Preferential Bayesian optimization allows optimization of objectives that are either expensive or difficult to measure directly, by relying on a minimal number of comparative evaluations done by a human expert. Generating candidate solutions for evaluation is also often expensive, but this cost is ignored by existing methods. We generalize preference-based optimization to explicitly account for production and evaluation costs with Consecutive Preferential Bayesian Optimization, reducing production cost by constraining comparisons to involve previously generated candidates. We also account for the perceptual ambiguity of the oracle providing the feedback by incorporating a Just-Noticeable Difference threshold into a probabilistic preference model to capture indifference to small utility differences. We adapt an information-theoretic acquisition strategy to this setting, selecting new configurations that are most informative about the unknown optimum under a preference model accounting for the perceptual ambiguity. We empirically demonstrate a notable increase in accuracy in setups with high production costs or with indifference feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。