arXiv:2505.00822stat.MEcs.LG2025-05被引 1

用聚类自助法改进强化学习,更可靠地发现干预关键变量。

Q-Learning with Clustered-SMART (cSMART) Data: Examining Moderators in the Construction of Clustered Adaptive Interventions

  • 基于聚类自助法的Q-learning框架,处理复杂数据结构
  • 在非正则条件下仍能保证置信区间覆盖率接近理论值
  • 适合研究团队构建个性化临床干预方案

聚类自适应干预(cAI)是预先设定的决策规则序列,指导从业者根据特定指标在群体层面调整干预以改善个体结果。聚类序贯多分配随机试验(cSMART)用于实证开发cAI。cSMART常见次级目标是评估候选分层变量对因果效应的调节作用。本文引入一种结合M-out-of-N聚类自助法的聚类Q-learning框架,利用cSMART数据评估一组候选分层变量是否有助于构建最优cAI。该方法可在存在“非正则性”这一经典挑战时,仍构建近似名义覆盖率的置信区间,实现对调节效应函数参数的可靠推断。模拟实验检验了该方法在不同非正则条件下的表现,并考察了聚类数量和组内相关系数对置信区间覆盖率的影响。方法应用于ADEPT数据集,以支持针对情绪障碍治疗中循证实践提升的诊所级cAI构建。

原文摘要 · Abstract (English)

A clustered adaptive intervention (cAI) is a pre-specified sequence of decision rules that guides practitioners on how best - and based on which measures - to tailor cluster-level intervention to improve outcomes at the level of individuals within the clusters. A clustered sequential multiple assignment randomized trial (cSMART) is a type of trial that is used to inform the empirical development of a cAI. The most common type of secondary aim in a cSMART focuses on assessing causal effect moderation by candidate tailoring variables. We introduce a clustered Q-learning framework with the M-out-of-N Cluster Bootstrap using data from a cSMART to evaluate whether a set of candidate tailoring variables may be useful in defining an optimal cAI. This approach could construct confidence intervals (CI) with near-nominal coverage to assess parameters indexing the causal effect moderation function. Specifically, it allows reliable inferences concerning the utility of candidate tailoring variables in constructing a cAI that maximizes a mean end-of-study outcome even when "non-regularity", a well-known challenge exists. Simulations demonstrate the numerical performance of the proposed method across varying non-regularity conditions and investigate the impact of varying number of clusters and intra-cluster correlation coefficient on CI coverage. Methods are applied on ADEPT dataset to inform the construction of a clinic-level cAI for improving evidence-based practice in treating mood disorders.

强化学习临床干预统计推断聚类数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。