arXiv:2410.10286cs.CLcs.IR2024-10

自动化生成阿拉伯文书籍索引,准确率高达96.6%

Back-of-the-Book Index Automation for Arabic Documents

  • 通过词性分析提取页面名词短语,构建向量库匹配索引项
  • 综合精确匹配、词汇与语义相似度,得分最高者为正确条目
  • 方法在阿拉伯文文档上达F1=0.966,适合出版与文献整理场景

书后索引对书籍可读性至关重要,但人工制作费时易错。本文针对阿拉伯文书籍,提出自动化索引项定位方法:从相关页段落中提取所有可能的名词短语(基于词性分析),存入向量数据库;再通过精确匹配、词汇相似度与语义相似度三项指标综合评分,选取最高分候选作为该术语的正确出现位置。经微调的启发式方法在测试中达到F1分数0.966(精确率=0.966,召回率=0.966)。结果表明该技术可显著简化索引创建与校验流程,为未来全自动索引生成奠定基础。

原文摘要 · Abstract (English)

Back-of-the-book indexes are crucial for book readability. Their manual creation is laborious and error prone. In this paper, we consider automating back-of-the-book index extraction for Arabic books to help simplify both the creation and review tasks. Given a back-of-the-book index, we aim to check and identify the accurate occurrences of index terms relative to the associated pages. To achieve this, we first define a pool of candidates for each term by extracting all possible noun phrases from paragraphs appearing on the relevant index pages. These noun phrases, identified through part-of-speech analysis, are stored in a vector database for efficient retrieval. We use several metrics, including exact matches, lexical similarity, and semantic similarity, to determine the most appropriate occurrence. The candidate with the highest score based on these metrics is chosen as the occurrence of the term. We fine-tuned a heuristic method, that considers the above metrics and that achieves an F1-score of .966 (precision=.966, recall=.966). These excellent results open the door for future work related to automation of back-of-the-book index generation and checking.

索引自动化阿拉伯文处理信息检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。