将调查题项自动链接到知识库,提升社科研究的可比性与可追溯性。
Enriching Social Science Research via Survey Item Linking
- 分两阶段:先识别文本中对题项的隐含提及,再匹配到知识库中的具体题项
- 构建包含20,454条句子的高质量双语数据集,支持任务评估
- 跨句上下文理解是难点,未来可尝试端到端模型优化
社会科学研究中,调查题项用于测量诸如生活满意度等潜在概念。研究人员通常不直接引用题项,而是通过转述方式在文中提及,这使得查找相关题项困难。本文提出调查题项链接(Survey Item Linking, SIL)任务,旨在自动识别文本中对题项的隐含提及并匹配至知识库中的对应项。该任务分为两个阶段:提及检测与实体消歧。由于现有数据集规模小、质量差,我们构建了一个包含20,454条英文和德文句子的高质量标注数据集,并独立与串联评估深度学习模型。实验表明任务可行,但第一阶段错误会传递至第二阶段,影响整体性能。尤其当提及需依赖多句上下文时,模型表现更差。未来可通过建模全文上下文、采用端到端系统、扩充多样化数据及提升知识库质量来改善。数据与代码已公开于 https://github.com/e-tornike/SIL。
原文摘要 · Abstract (English)
Questions within surveys, called survey items, are used in the social sciences to study latent concepts, such as the factors influencing life satisfaction. Instead of using explicit citations, researchers paraphrase the content of the survey items they use in-text. However, this makes it challenging to find survey items of interest when comparing related work. Automatically parsing and linking these implicit mentions to survey items in a knowledge base can provide more fine-grained references. We model this task, called Survey Item Linking (SIL), in two stages: mention detection and entity disambiguation. Due to an imprecise definition of the task, existing datasets used for evaluating the performance for SIL are too small and of low-quality. We argue that latent concepts and survey item mentions should be differentiated. To this end, we create a high-quality and richly annotated dataset consisting of 20,454 English and German sentences. By benchmarking deep learning systems for each of the two stages independently and sequentially, we demonstrate that the task is feasible, but observe that errors propagate from the first stage, leading to a lower overall task performance. Moreover, mentions that require the context of multiple sentences are more challenging to identify for models in the first stage. Modeling the entire context of a document and combining the two stages into an end-to-end system could mitigate these problems in future work, and errors could additionally be reduced by collecting more diverse data and by improving the quality of the knowledge base. The data and code are available at https://github.com/e-tornike/SIL .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。