选词速度是脑机接口关键指标，该研究找出了最能反映它的评估方法。
Which Metric Reflects the Spelling Rate Accuracy in Event-Related Potential-Based Brain-Computer Interfaces?
- 对比13种评估指标与选词速度的相关性，识别出最优指标
- 发现贝叶斯评分、MCC和处理类别不平衡的指标更可靠
- 适合关注真实性能评估的脑机接口研究者参考
在同步脑机接口系统中，常用性能指标如损失和准确率对多数范式有效；但对事件相关电位（ERP）应用而言，选词速度（即正确选择字符数）更为重要，因为它直接影响信息传输率（ITR）等性能指标。此外，基于ERP的脑机接口存在类别分布不均问题，需采用能处理不平衡数据的指标，如受试者工作特征曲线下面积（ROC AUC）。本研究通过两个数据集（私有LARESI ERP数据集和公开OpenBMI ERP数据集）分析了13个指标与选词速度的相关性，结果表明：贝叶斯评分、马修斯相关系数（MCC）以及考虑类别不平衡的指标（包括ROC AUC、PR曲线下面积（PR AUC）、平均精度（AP）和部分AUC（pAUC））表现更优。研究建议在ERP基脑机接口实验中报告这些指标。
原文摘要 · Abstract (English)
For predictive models, the often-reported performance metrics are the loss and accuracy. In synchronous Brain- Computer Interface (BCI) systems, these metrics are informative for most BCI paradigms; however, for Event-Related Potential (ERP) applications the spelling rate, which measures the number of characters correctly selected is more important as it influences the estimation of information transfer rate (ITR) and any related metric measuring spelling performance. Moreover, ERP-based BCIs hold imbalanced data class distributions, which require reporting metrics that can handle the imbalance, such as the area under the receiver operating characteristic curve (ROC AUC). In this work, we study the correlation of the spelling rate with 13 metrics to identify which among them best reflect user spelling performance and how they are affected by trial repetition. The Results of two datasets (a private LARESI ERP dataset and the public OpenBMI ERP dataset) favor the Brier score, Matthews Correlation Coefficient (MCC), and the metrics that account for class imbalance in binary classification: ROC AUC, area under the Precision-Recall curve (PR AUC), Average Precision (AP), and partial AUC (pAUC). These findings encourage researchers and practitioners to report those metrics in ERP-based BCI experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。