arXiv:2606.28620cs.IRcs.CY2026-06中稿 · Transactions on Ma…

复现FACTER框架发现动态修复机制收益有限,静态指令已可达到相似公平性效果。

Reproducing FACTER: Fairness via Conformal Thresholding and Prompt Repair

  • 采用固定候选集的重排序设置,对比动态修复与静态指令
  • 动态修复虽降低自适应阈值违规次数,但全局公平性未显著提升
  • 适合关注大模型推荐中公平性实现成本的研究者

Fayyazi 等人(2025)提出了 FACTER,一种模型无关的框架,通过置信区间阈值和迭代提示修复,在 LLM 推荐中同时保障公平性与统计覆盖。本文对 FACTER 在多种架构和数据稀疏度下的可复现性进行了评估,涵盖原始开放式生成任务及受限重排序扩展。严格复现下观察到推荐效用差异,溯源为原研究目标集评估不明确。随后在固定候选集设定下评估 FACTER,引入静态零样本公平性基线以分离迭代提示修复的贡献。分析表明,FACTER 始终减少自适应阈值违规次数,但此类改进在固定阈值或全局公平性指标上未持续体现。在受限重排序设置中,静态公平性指令即可达到与 FACTER 动态修复相当的语义一致性,暗示在线修复机制在此形式下收益有限。所有代码与复现材料见 https://github.com/oscar-omlf/facter-repr。

原文摘要 · Abstract (English)

Fayyazi et al. (2025) recently proposed FACTER, a model-agnostic framework designed to jointly enforce fairness and statistical coverage in LLM-based recommendation through conformal thresholding and iterative prompt repair. In this work, we conduct a reproducibility study of the FACTER framework across diverse architectures and dataset sparsity levels, evaluating both the original open-ended generation task and a constrained re-ranking extension. Under the strict reproduction, we observe a divergence in recommendation utility, which we trace to underspecified target-set evaluation in the original study. We then use the constrained re-ranking setting to evaluate FACTER when the candidate set is fixed, and introduce a static Fair Zero-Shot baseline to isolate the contribution of the iterative prompt repair loop. Our analysis shows that FACTER consistently reduces adaptive-threshold violation counts, but that these reductions are not consistently reflected under the fixed threshold or in global fairness metrics. In the constrained ranking setting, static fairness instructions achieve comparable semantic-parity outcomes to FACTER's dynamic repair loop, suggesting that the additional online repair mechanism provides limited benefit in this formulation. All code and reproduction artifacts are available at https://github.com/oscar-omlf/facter-repr.

公平性大模型推荐可复现性提示修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。