人机协作自进化系统，提升主题分析的准确与效率
CentaurTA Studio: A Self-Improving Human-Agent Collaboration System for Thematic Analysis
- 分两阶段反馈：模拟草稿+专家验证，提升可控性
- 92.12%准确率，10轮内达峰值，比人工快25分钟
- 支持可解释评估，适合需要高可靠质性研究的团队
主题分析难以规模化：人工流程耗时，全自动系统又缺乏可控性与透明评估。我们提出基于网页的自进化人机协作系统 CentaurTA Studio，用于开放式编码与主题构建。系统集成三项核心机制：（1）分阶段人类反馈流程，分离模拟器草稿与专家验证；（2）持续提示优化，将已验证反馈提炼为可复用对齐原则；（3）基于评分标准的评估与早期停止机制，实现过程控制。在三个领域中，CentaurTA 在开放式编码与主题构建任务上均达到最优表现，最高准确率达92.12%，显著优于基线系统。评分标准驱动的 LLM 判官与人工标注者间一致性达到实质性水平（平均 κ=0.68）。消融实验表明，移除反馈环使性能从90%降至81%，取消批判模块或早期停止会降低准确率或增加交互成本。完整系统在10轮迭代内（约25分钟）达到峰值性能，效率优于纯人工精修。
原文摘要 · Abstract (English)
Thematic analysis is difficult to scale: manual workflows are labor-intensive, while fully automated pipelines often lack controllability and transparent evaluation. We present \textbf{CentaurTA Studio}, a web-based system for self-improving human--agent collaboration in open coding and theme construction. The system integrates (1) a two-stage human feedback pipeline separating simulator drafting and expert validation, (2) persistent prompt optimization that distills validated feedback into reusable alignment principles, and (3) rubric-based evaluation with early stopping for process control. Across three domains, CentaurTA achieves the strongest performance in both Open Coding and Theme Construction, reaching up to 92.12\% accuracy and consistently outperforming baseline systems. Agreement between the rubric-based LLM judge and human annotators reaches substantial reliability (average $κ= 0.68$). Ablation studies show that removing the feedback loop reduces performance from 90\% to 81\%, while eliminating the Critic or early stopping degrades accuracy or increases interaction cost. The full system reaches peak performance within 10 iterative rounds (about 25 minutes), demonstrating improved efficiency over expert-only refinement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。