arXiv:2608.27809cs.IR2026-08

用聊天记录做个人记忆检索,提升大模型问答精准度。

LINE Conversation History Retrieval for Personal Memory RAG: Evaluating Search Representations and Hybrid Retrieval

  • 构建三种文本表示:原始、摘要、摘要+原文片段组合
  • 混合检索使召回率达0.697,比单一方法高11.3%
  • 适合研究个性化记忆增强生成的学者与开发者

为探索大语言模型(LLM)的个人记忆增强生成(RAG),本研究以单用户LINE聊天记录为对象,开展仅检索的案例分析。将358,896条消息分割为22,329个时间连贯的文本块,构建三种搜索表示:raw_text、生成摘要和embedding_text(结合摘要、原始文本片段及其他固定文本)。在100个经单名标注者验证的问题上,对比了BM25、密集向量检索及线性混合检索。单独检索中,embedding_text_bm25表现最佳,Recall@5为0.584。进一步测试6种检索器组合与21种权重,共126种配置。最终选定embedding_text_bm25与embedding_text_vector在beta=0.45时,取得Recall@5=0.697、MRR@5=0.595、nDCG@5=0.575。其召回率相比embedding_text_bm25提升0.113,95%置信区间[0.048, 0.184],表明显著提升。与beta=0.50的摘要混合方案相比,差异为0.050,置信区间[-0.013, 0.115],无显著差异。17个聚合问题结果偏低,表明块级检索在证据分散于多时段多对话时存在困难。该评估基于同一问题集进行配置搜索,未涉及最终答案生成或对新问题的泛化能力。

原文摘要 · Abstract (English)

As an initial step toward personal memory retrieval-augmented generation (RAG) for large language models (LLMs), this study presents a retrieval-only case study over one user's LINE conversation history. We segmented 358,896 messages into 22,329 temporally coherent chunks and constructed three search representations: raw_text, a generated summary, and embedding_text, which combines a summary with a raw-text excerpt and other fixed text. We compared BM25, dense vector retrieval, and linear hybrid retrieval on 100 evaluation questions verified by a single annotator. Among individual retrievers, embedding_text_bm25 achieved the highest point estimate, with Recall@5 of 0.584. We then explored six retriever pairings and 21 weights, for 126 configurations on the same evaluation set. The selected combination of embedding_text_bm25 and embedding_text_vector at beta = 0.45 achieved Recall@5 = 0.697, MRR@5 = 0.595, and nDCG@5 = 0.575. Its Recall@5 exceeded that of embedding_text_bm25 by 0.113, with a question-level paired percentile-bootstrap 95% confidence interval of [0.048, 0.184]. This interval is conditional on fixing the configuration selected on the same 100 questions and does not account for uncertainty from configuration selection or weight search. The difference from a summary-based hybrid at beta = 0.50 was 0.050, with a 95% confidence interval of [-0.013, 0.115], so no clear difference could be established. The 17 aggregate questions also yielded lower point estimates than the other question types, suggesting that flat chunk-level retrieval struggles when evidence is distributed across multiple times and conversations. This evaluation is an exploratory single-user, single-annotator study conducted on the same question set used for configuration search; it does not evaluate final answer generation or generalization to unseen questions.

个人记忆检索增强聊天数据混合检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。