用提示词与数学模型结合，提升混合语言对话的信息检索准确率
RetrieveGPT: Merging Prompts and Mathematical Models for Enhanced Code-Mixed Information Retrieval
- 结合GPT-3.5 Turbo提示词与文档顺序构建数学模型
- 在罗马字母转写孟加拉语+英语数据上实现高相关度文档识别
- 适合研究多语言社交文本处理的NLP研究人员
代码混用是多语言社会中普遍存在的语言现象，尤其在印度，社交媒体用户常以罗马字母转写孟加拉语与英语混合表达。本文针对此类混合语言对话中信息提取的挑战，提出一种新方法，自动识别相关答案。基于来自Facebook的查询与文档数据及查询相关性文件（QRels），实验采用GPT-3.5 Turbo通过提示工程，并结合相关文档的序列特性，构建数学模型以检测与查询匹配的文档。结果表明，该方法在复杂、非正式的混合语言数字对话中有效提取相关信息，推动了自然语言处理在多语言非正式文本环境中的应用。
原文摘要 · Abstract (English)
Code-mixing, the integration of lexical and grammatical elements from multiple languages within a single sentence, is a widespread linguistic phenomenon, particularly prevalent in multilingual societies. In India, social media users frequently engage in code-mixed conversations using the Roman script, especially among migrant communities who form online groups to share relevant local information. This paper focuses on the challenges of extracting relevant information from code-mixed conversations, specifically within Roman transliterated Bengali mixed with English. This study presents a novel approach to address these challenges by developing a mechanism to automatically identify the most relevant answers from code-mixed conversations. We have experimented with a dataset comprising of queries and documents from Facebook, and Query Relevance files (QRels) to aid in this task. Our results demonstrate the effectiveness of our approach in extracting pertinent information from complex, code-mixed digital conversations, contributing to the broader field of natural language processing in multilingual and informal text environments. We use GPT-3.5 Turbo via prompting alongwith using the sequential nature of relevant documents to frame a mathematical model which helps to detect relevant documents corresponding to a query.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。