用脑电波直接检索文本,省去打字步骤
Towards Brain Passage Retrieval -- An Investigation of EEG Query Representations
- 用脑电图信号直接匹配文本段落,跳过文字输入
- 在30人实验中,精确度比现有方法高8.81%
- 适合无法打字或信息需求模糊的用户
信息检索系统依赖用户将内在需求转化为文本查询,但这一过程常因认知负担和表达不准确而失败,尤其对信息需求模糊或有肢体障碍的用户更为显著。近年研究尝试通过脑机接口(BMI)直接从脑信号中解码查询,但现有方法在学习鲁棒的脑-文本表示方面效果有限,难以捕捉脑信号中的语义细节。为此,我们提出BPR(Brain Passage Retrieval)框架,摒弃中间查询翻译环节,实现从脑电图(EEG)信号直接检索相关文本段落。该方法利用密集检索架构,将EEG信号与文本段落映射至共享语义空间。在ZuCo数据集上的实验表明,BPR在precision@5上相较现有EEG-to-text基线提升最高达8.81%,且在30名参与者中保持稳定性能。消融实验证明,硬负样本采样和专用脑信号编码器对跨模态对齐至关重要。结果证实了直接脑-段落检索的可行性,为构建更自然的认知-检索接口奠定基础。
原文摘要 · Abstract (English)
Information Retrieval (IR) systems primarily rely on users' ability to translate their internal information needs into (text) queries. However, this translation process is often uncertain and cognitively demanding, leading to queries that incompletely or inaccurately represent users' true needs. This challenge is particularly acute for users with ill-defined information needs or physical impairments that limit traditional text input, where the gap between cognitive intent and query expression becomes even more pronounced. Recent neuroscientific studies have explored Brain-Machine Interfaces (BMIs) as a potential solution, aiming to bridge the gap between users' cognitive semantics and their search intentions. However, current approaches attempting to decode explicit text queries from brain signals have shown limited effectiveness in learning robust brain-to-text representations, often failing to capture the nuanced semantic information present in brain patterns. To address these limitations, we propose BPR (Brain Passage Retrieval), a novel framework that eliminates the need for intermediate query translation by enabling direct retrieval of relevant passages from users' brain signals. Our approach leverages dense retrieval architectures to map EEG signals and text passages into a shared semantic space. Through comprehensive experiments on the ZuCo dataset, we demonstrate that BPR achieves up to 8.81% improvement in precision@5 over existing EEG-to-text baselines, while maintaining effectiveness across 30 participants. Our ablation studies reveal the critical role of hard negative sampling and specialised brain encoders in achieving robust cross-modal alignment. These results establish the viability of direct brain-to-passage retrieval and provide a foundation for developing more natural interfaces between users' cognitive states and IR systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。