对比两种增强方法在小模型上的表现，发现RAG更适合作为本地个人助手。
Assessing RAG and HyDE on 1B vs. 4B-Parameter Gemma LLMs for Personal Assistants Integretion
- 用RAG和HyDE提升小型Gemma模型的问答能力
- RAG降低17%延迟且消除事实幻觉，HyDE则增加25%-40%耗时
- 小模型上RAG更稳定，适合隐私敏感的本地部署
资源效率是大型语言模型在边缘及隐私敏感场景中部署的关键障碍。本研究评估了两种增强策略——检索增强生成（RAG）与假设文档嵌入（HyDE）——在10亿参数和40亿参数小型Gemma LLMs上的效果，应用于以隐私为先的个人助理系统。通过MongoDB实现短期记忆，利用Qdrant进行长期语义存储，由FastAPI与LangChain协同调度，并通过React.js前端暴露服务。在两个模型规模下，RAG均将响应延迟降低最多17%，并消除用户特定与领域特定查询中的事实性幻觉；而HyDE虽提升复杂物理提示的语义相关性，但导致响应时间增加25%至40%，并在个人数据检索中出现显著幻觉。对比1B与4B模型，扩大规模对基线和RAG流水线仅带来微弱吞吐量提升，却显著放大HyDE的计算开销与波动性。研究结果表明，对于基于小型LLM的本地化个人助理，RAG是更务实的选择。
原文摘要 · Abstract (English)
Resource efficiency is a critical barrier to deploying large language models (LLMs) in edge and privacy-sensitive applications. This study evaluates the efficacy of two augmentation strategies--Retrieval-Augmented Generation (RAG) and Hypothetical Document Embeddings (HyDE)--on compact Gemma LLMs of 1 billion and 4 billion parameters, within the context of a privacy-first personal assistant. We implement short-term memory via MongoDB and long-term semantic storage via Qdrant, orchestrated through FastAPI and LangChain, and expose the system through a React.js frontend. Across both model scales, RAG consistently reduces latency by up to 17\% and eliminates factual hallucinations when responding to user-specific and domain-specific queries. HyDE, by contrast, enhances semantic relevance--particularly for complex physics prompts--but incurs a 25--40\% increase in response time and a non-negligible hallucination rate in personal-data retrieval. Comparing 1 B to 4 B models, we observe that scaling yields marginal throughput gains for baseline and RAG pipelines, but magnifies HyDE's computational overhead and variability. Our findings position RAG as the pragmatic choice for on-device personal assistants powered by small-scale LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。