通过树结构共享反馈,让多策略测试成本降低至独立测试的1/J
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing

- 用可预测的树结构耦合策略历史,实现精确反馈共享
- 实际奖励查询数期望为T加树边总变差,远低于独立测试的JT
- 适合需要高效评估多个自适应算法的平台场景
在线平台需频繁比较多个自适应决策策略(如推荐系统、定价规则、语言模型代理),而每次有回报的交互成本高或风险大。传统A/B/n测试为每个J≥2个策略分配独立的长度为T的轨迹,共需JT次结果。本文提出树耦合A/B测试( TCAB),一种适用于任意依赖历史的上下文-赌博机策略的精确反馈共享设计。每轮中,一个可预测的树连接各策略的历史;每个父-子节点的上下文-动作分布被最大耦合,每对匹配的树边共享一次奖励。各策略仍保持其独立有限时域轨迹分布,尽管策略间被有意关联。若D_{e,t}表示第t轮在树边e上的不匹配,奖励查询总数满足路径恒等式N(T)=T+∑_{t,e}D_{e,t},因此期望值等于T加上树边累积总变差。该成本在选定树结构下条件最优,且当前轮次的最小生成树在树设计中近似最优。固定J时,所有策略具有次线性伪遗憾,且几乎必然存在唯一最优动作,故E[N(T)]=T+o(T),远优于独立测试的JT。还获得了成对策略对比的有限样本方差界。在奖励模型评估、多选语言模型评估和自适应搜索策略实验中,均显著提升成本-精度权衡表现。
原文摘要 · Abstract (English)
Online platforms increasingly compare many adaptive decision policies---ranking systems, recommendation algorithms, pricing rules, and language-model agents---while each reward-bearing interaction can be costly or risky. A direct A/B/n design gives each of $J\ge 2$ policies its own horizon-$T$ trajectory and therefore uses $JT$ outcomes. We introduce Tree-Coupled A/B Testing (\TCAB), an exact feedback-sharing design for arbitrary history-dependent contextual-bandit policies. At each round, a predictable tree connects the current policy histories; every parent--child context--action law is maximally coupled, and one reward is shared within each component of matched tree edges. Every policy retains exactly its standalone finite-horizon trajectory law, even though the policies are deliberately dependent. If $D_{e,t}$ records a mismatch on tree edge $e$ at round $t$, the number of reward queries satisfies the pathwise identity $N(T)=T+\sum_{t,e}D_{e,t}$ and hence equals $T$ plus cumulative tree-edge total variation in expectation. This cost is conditionally optimal among exact edge-local designs on the selected tree, and a current-round minimum-spanning tree is myopically optimal among tree designs. For fixed $J$, sublinear pseudo-regret of every policy and almost-sure uniqueness of the oracle action imply $\mathbb{E}[N(T)]=T+o(T)$, versus $JT$ for independent runs. We also obtain finite-sample variance bounds for pairwise policy contrasts. Experiments on reward-model evaluation, multiple-choice language-model evaluation, and adaptive search policies demonstrate substantial improvements in the cost--precision frontier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。