用大模型排序技术,快速从海量代码中找出最可能有漏洞的函数。
Sift or Get Off the PoC: Applying Information Retrieval to Vulnerability Research with SiftRank
- 用大模型批量排序代码片段,动态停止评估以节省计算。
- 99秒内从2197个函数中定位到漏洞修复点,成本仅0.82美元。
- 无需训练或专用模型,普通API即可部署,适合安全研究人员使用。
安全研究本质上是资源受限下的优先级问题:攻击面庞大而分析时间有限。顶尖研究人员往往擅长直觉判断应优先分析哪些部分。本文将这一选择难题重新定义为信息检索问题,利用大模型作为通用排序器,提出SiftRank算法。该算法通过三种机制实现线性复杂度:小批次(约10项)的列表排序、基于评分分布稳定的自适应终止策略,以及对高相关文档的迭代聚焦。与需先筛选至约100候选的传统重排序方法不同,SiftRank可直接处理数千项,每项在多个随机批次中评估,以缓解大模型判断不一致问题。在真实N-day漏洞分析中,成功于99秒内从2,197个变更函数中识别出漏洞修复函数,推理成本仅0.82美元。本方法支持可扩展的安全优先级决策,仅需标准大模型API,无需特殊基础设施或领域微调。开源实现见https://github.com/noperator/siftrank。
原文摘要 · Abstract (English)
Security research is fundamentally a problem of resource constraint and consequent prioritization. There is simply too much attack surface and too little time and energy to spend analyzing it all. The most effective security researchers are often those who are most skilled at intuitively deciding which part of an expansive attack surface to investigate. We demonstrate that this problem of selecting the most promising option from among many possibilities can be reframed as an information retrieval problem, and solved using document ranking techniques with LLMs performing the heavy lifting as general-purpose rankers. We present SiftRank, a ranking algorithm achieving O(n) complexity through three key mechanisms: listwise ranking using an LLM to order documents in small batches of approximately 10 items at a time; inflection-based convergence detection that adaptively terminates ranking when score distributions have stabilized; and iterative refinement that progressively focuses ranking effort on the most relevant documents. Unlike existing reranking approaches that require a separate first-stage retrieval step to narrow datasets to approximately 100 candidates, SiftRank operates directly on thousands of items, with each document evaluated across multiple randomized batches to mitigate inconsistent judgments by an LLM. We demonstrate practical effectiveness on N-day vulnerability analysis, successfully identifying a vulnerability-fixing function among 2,197 changed functions in a stripped binary firmware patch within 99 seconds at an inference cost of $0.82. Our approach enables scalable security prioritization for problems that are generally constrained by manual analysis, requiring only standard LLM API access without specialized infrastructure, embedding, or domain-specific fine-tuning. An open-source implementation of SiftRank may be found at https://github.com/noperator/siftrank.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。