通过追踪费舍尔信息，用最少问题实现高效自适应测试
On the Optimality of Tracking Fisher Information in Adaptive Testing with Stochastic Binary Responses
- 自适应选择题目以最大化费舍尔信息，动态优化测试过程
- 在固定置信度和固定预算下均达到理论最优性能
- 适合需要高效准确评估能力的在线测试与推荐系统
我们研究从序列二元响应中估计连续能力参数的问题，通过主动选择不同难度的问题进行自适应测试，该场景广泛存在于自适应测评与在线偏好学习中。目标是在尽可能少的提问次数下，确保估计值落在预定误差范围内。本文提出一种简单算法：通过最大化费舍尔信息来自适应选择问题，并采用矩估计法更新估计值，同时设计一种新检验统计量以判断估计是否足够准确。理论上证明，这种费舍尔信息追踪策略在固定置信度和固定预算两种典型设置下均达到最优性能。分析克服了固定预算情形下的关键技术难点——处理估计值演化与查询分布间的依赖关系，通过利用模型的结构对称性，并结合大偏差工具与Ville不等式。结果为简单高效的自适应测试方法提供了严格的理论支撑。
原文摘要 · Abstract (English)
We study the problem of estimating a continuous ability parameter from sequential binary responses by actively asking questions with varying difficulties, a setting that arises naturally in adaptive testing and online preference learning. Our goal is to certify that the estimate lies within a desired margin of error, using as few queries as possible. We propose a simple algorithm that adaptively selects questions to maximize Fisher information and updates the estimate using a method-of-moments approach, paired with a novel test statistic to decide when the estimate is accurate enough. We prove that this Fisher-tracking strategy achieves optimal performance in both fixed-confidence and fixed-budget regimes, which are commonly invested in the best-arm identification literature. Our analysis overcomes a key technical challenge in the fixed-budget setting -- handling the dependence between the evolving estimate and the query distribution -- by exploiting a structural symmetry in the model and combining large deviation tools with Ville's inequality. Our results provide rigorous theoretical support for simple and efficient adaptive testing procedures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。