arXiv:2603.17386cs.IRcs.CL2026-03

PJB基准评估人岗匹配中的推理能力,揭示系统失败原因。

PJB: A Reasoning-Aware Benchmark for Person-Job Retrieval

  • 用完整职位描述作查询,简历作文档,基于胜任力判断相关性。
  • 跨行业表现差异大,模块升级收益远小于领域差异影响。
  • 提供诊断标签,帮助定位系统瓶颈,指导优化投入方向。

随着检索模型在通用基准上趋于收敛,关键问题已从‘谁得分更高’转向‘系统在何处失效,为何失效’。人岗匹配任务亟需此类诊断能力——不仅需验证显式约束,还需完成技能迁移推理与岗位胜任力推断,但现有基准缺乏系统性诊断支持。我们提出PJB(Person-Job Benchmark),一个面向推理的检索评估数据集:以完整职位描述为查询,完整简历为文档,通过岗位胜任力判断定义相关性,基于涵盖六个行业领域、近20万份简历的真实招聘数据构建,并引入领域族与推理类型诊断标签,将评估从‘谁得分高’升级为‘系统差异在哪,为何不同’。密集检索诊断实验显示,跨行业性能异质性远超同一模型模块升级带来的增益,表明仅看平均分会严重误导优化决策。模块层面,重排序稳定提升性能,而查询理解与重排序结合反而导致整体性能下降——二者面临根本不同的改进瓶颈。PJB的价值不在于新的排行榜,而在于为招聘检索系统提供一张能力地图,精准定位优化投资点。

原文摘要 · Abstract (English)

As retrieval models converge on generic benchmarks, the pressing question is no longer "who scores higher" but rather "where do systems fail, and why?" Person-job matching is a domain that urgently demands such diagnostic capability -- it requires systems not only to verify explicit constraints but also to perform skill-transfer inference and job-competency reasoning, yet existing benchmarks provide no systematic diagnostic support for this task. We introduce PJB (Person-Job Benchmark), a reasoning-aware retrieval evaluation dataset that uses complete job descriptions as queries and complete resumes as documents, defines relevance through job-competency judgment, is grounded in real-world recruitment data spanning six industry domains and nearly 200,000 resumes, and upgrades evaluation from "who scores higher" to "where do systems differ, and why" through domain-family and reasoning-type diagnostic labels. Diagnostic experiments using dense retrieval reveal that performance heterogeneity across industry domains far exceeds the gains from module upgrades for the same model, indicating that aggregate scores alone can severely mislead optimization decisions. At the module level, reranking yields stable improvements while query understanding not only fails to help but actually degrades overall performance when combined with reranking -- the two modules face fundamentally different improvement bottlenecks. The value of PJB lies not in yet another leaderboard of average scores, but in providing recruitment retrieval systems with a capability map that pinpoints where to invest.

人岗匹配推理评估诊断分析检索基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。