用户提前预判他人行为,集体协作可提升系统表现
Look-Ahead Reasoning on Learning Platforms
- 引入层级思维模型,模拟用户预判同伴行为的策略
- 集体协作能改善系统结果,但个体收益不随推理层级提升
- 提出学习者与用户目标对齐的新概念,适用于制度设计
在众多学习平台中,模型训练的优化目标反映设计者偏好,而非实际使用者利益,导致用户可能采取策略性行为以获取更优结果。以往研究多关注用户对已部署模型的反应,未考虑用户间行为的相互影响。本文提出前瞻性推理框架,其中用户基于他人行动对未来预测进行预判。首先形式化层次化思维(level-k thinking),表明尽管收敛加速,但最终均衡不变,高阶推理对个体无长期收益。随后研究集体推理,即用户通过协调行动优化对模型的共同影响。对比集体与自私行为,揭示协作的效益与局限,并提出学习者与用户效用对齐这一关键概念。前瞻性推理可视为算法集体行动的泛化,首次刻画了对抗算法系统时协作带来的效用权衡。
原文摘要 · Abstract (English)
On many learning platforms, the optimization criteria guiding model training reflect the priorities of the designer rather than those of the individuals they affect. Consequently, users may act strategically to obtain more favorable outcomes. While past work has studied strategic user behavior on learning platforms, the focus has largely been on strategic responses to a deployed model, without considering the behavior of other users. In contrast, look-ahead reasoning takes into account that user actions are coupled, and -- at scale -- impact future predictions. Within this framework, we first formalize level-k thinking, a concept from behavioral economics, where users aim to outsmart their peers by looking one step ahead. We show that, while convergence to an equilibrium is accelerated, the equilibrium remains the same, providing no benefit of higher-level reasoning for individuals in the long run. Then, we focus on collective reasoning, where users take coordinated actions by optimizing through their joint impact on the model. By contrasting collective with selfish behavior, we characterize the benefits and limits of coordination; a new notion of alignment between the learner's and the users' utilities emerges as a key concept. Look-ahead reasoning can be seen as a generalization of algorithmic collective action; we thus offer the first results characterizing the utility trade-offs of coordination when contesting algorithmic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。