解决大模型反馈数据偏差问题，用贝叶斯方法更准估算真实质量。
Correcting Selection Bias in Sparse User Feedback for Large Language Model Quality Estimation: A Multi-Agent Hierarchical Bayesian Approach
- 构建三代理的分层贝叶斯框架，不依赖个体标签。
- 在模拟数据上，偏差修正后误差仅4-13个百分点。
- 适合生产环境中的模型质量监控与在线调整。
大型语言模型在实际部署中收到的用户反馈来自非随机群体：点赞集中在满意度分布的尾部，直接平均会导致估计值与真实质量相差40-50个百分点。本文将此视为主题与情感分层的样本选择偏差问题，提出一个三代理分层贝叶斯流程，无需个体交互的真值标签。主题聚类代理通过UMAP + HDBSCAN对文本嵌入进行分区；偏倚建模代理在NUTS下拟合两阶段分层贝塔-二项模型，利用部分共用信息推断各主题的选择率 $s_c$ 和质量 $q_c$；合成代理则根据真实主题频率 $ar{π}_c = n_c/N$ 对 $q_c$ 加权，输出修正后的聚合后验质量 $ar{Q} = igsum_c ar{π}_c q_c$ 及可信区间，并提供漂移信号用于在线再校准。使用UltraFeedback数据集（保留10,232条交互，$C=18$个聚类，$Q^ ext{⋆}=0.6249$）进行验证，模拟了主题和情感相关的反馈偏差。对比五种贝叶斯变体与朴素均值、逆概率加权基线。在反馈通道上施加温和先验（典型正向反馈率与负向/正向比值，可从任意生产仪表盘获取，无需标签），分层知情方法在偏差比1:1至30:1范围内保持与$Q^ ext{⋆}$相差4-13个百分点，且95%可信区间在50/50随机种子重复实验中覆盖$Q^ ext{⋆}$的比例达50%（当$κ_{ ext{max}}=10$时）。若无通道先验，所有弱先验变体均偏离$Q^ ext{⋆}$ 22-33个百分点：因每簇充分统计量构成一参数族等效拟合，是通道先验而非质量先验打破退化问题。
原文摘要 · Abstract (English)
[Abridged] Production LLM deployments receive feedback from a non-random fraction of users: thumbs sit mostly in the tails of the satisfaction distribution, and a naive average over them can land 40-50 percentage points away from true system quality. We treat this as a topic- and sentiment- stratified selection-bias problem and propose a three-agent hierarchical Bayesian pipeline that does not require ground-truth labels on individual interactions. A Topic Clustering Agent partitions the stream via UMAP + HDBSCAN over text embeddings; a Bias Modeling Agent fits a two-stage hierarchical Beta-Binomial under NUTS, inferring per-topic selection rates $s_c$ and quality $q_c$ with partial pooling; a Synthesis Agent reweights $q_c$ by true topic prevalence $\hatπ_c = n_c/N$ to report a bias-corrected aggregate posterior $\bar Q = \sum_c \hatπ_c q_c$ with credible interval, plus drift signals for online recalibration. Validation uses UltraFeedback (N=10,232 retained interactions, $C=18$ clusters, $Q^\star=0.6249$) with simulated topic- and sentiment-dependent selection biases. We compare five Bayesian variants against Naive and IPW baselines. A mild prior on the feedback channel (typical positive-feedback rate and negative-to-positive ratio, both readable from any production dashboard without labels) keeps Hierarchical-Informed within 4-13 pp of $Q^\star$ as the bias ratio sweeps from 1:1 to 30:1, with 95% credible intervals covering $Q^\star$ in 50/50 random-seed replicates at $κ_{\max}=10$. Without channel-side priors, every weak-prior variant misses $Q^\star$ by 22-33 pp: the per-cluster sufficient statistics admit a one-parameter family of equally good fits, and the prior on the bias channel (not on latent quality) is what breaks the degeneracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。