让多目标离线强化学习自动平衡公平性，但实验需重做。
[Re] FairDICE: A Fair Tradeoff in Multi-objective Offline RL
- 基于OptiDICE改进，自动学习多目标权重以实现公平权衡
- 修复代码后可在复杂环境与高维奖励中稳定运行
- 适合关注多目标公平性的研究者，但需谨慎对待原实验结果
离线强化学习（Offline RL）仅通过示范数据训练策略。在涉及多目标的环境中，现有算法难以高效找到公平的折衷方案。FairDICE（参见arXiv:2506.08062v2）通过改进OptiDICE，自动学习多目标权重以激励目标间的公平性。本复现研究检验了FairDICE的可复现性。发现多数理论结论成立，但代码错误导致其在连续环境中退化为标准行为克隆，且部分关键超参数未明确定义。修正后，实验显示FairDICE可在复杂环境和高维奖励下扩展应用，但仍依赖在线超参数调优。结论：理论上有价值，但实验论证需大幅修订。
原文摘要 · Abstract (English)
Offline Reinforcement Learning (RL) is an emerging field of RL in which policies are learned solely from demonstrations. Within offline RL, some environments involve balancing multiple objectives, but existing multi-objective offline RL algorithms do not provide an efficient way to find a fair compromise. FairDICE (see arXiv:2506.08062v2) seeks to fill this gap by adapting OptiDICE (an offline RL algorithm) to automatically learn weights for multiple objectives to e.g. incentivise fairness among objectives. As this would be a valuable contribution, this replication study examines the replicability of claims made regarding FairDICE. We find that many theoretical claims hold, but an error in the code reduces FairDICE to standard behaviour cloning in continuous environments, and many important hyperparameters were originally underspecified. After rectifying this, we show in experiments extending the original paper that FairDICE can scale to complex environments and high-dimensional rewards, though it can be reliant on (online) hyperparameter tuning. We conclude that FairDICE is a theoretically interesting method, but the experimental justification requires significant revision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。