让机器学习评估指标同时考虑成本与收益，更贴近实际应用需求。
Cost and Reward Infused Metric Elicitation
- 扩展了基于混淆矩阵的评估方法，加入可量化成本与奖励
- 在合成数据上快速收敛到真实最优评估指标
- 适合关注模型部署成本与收益平衡的研究者
在机器学习中，度量提取旨在选择最能反映个体对特定应用隐含偏好的性能度量。现有方法仅依赖模型混淆矩阵中的准确率信息，忽略了实际可行性因素如不同成本或延迟。本文在Hiranandani等人的多分类度量提取框架基础上，将其提出的对角线线性性能度量提取（DLPME）算法拓展至包含有界成本与奖励。实验结果表明，该方法在合成数据上能够快速收敛至真实度量。
原文摘要 · Abstract (English)
In machine learning, metric elicitation refers to the selection of performance metrics that best reflect an individual's implicit preferences for a given application. Currently, metric elicitation methods only consider metrics that depend on the accuracy values encoded within a given model's confusion matrix. However, focusing solely on confusion matrices does not account for other model feasibility considerations such as varied monetary costs or latencies. In our work, we build upon the multiclass metric elicitation framework of Hiranandani et al., extrapolating their proposed Diagonal Linear Performance Metric Elicitation (DLPME) algorithm to account for additional bounded costs and rewards. Our experimental results with synthetic data demonstrate our approach's ability to quickly converge to the true metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。