用语义匹配检测网页链接失效,速度比大模型快47倍
SemLink: A Semantic-Aware Automated Test Oracle for Hyperlink Verification using Siamese Sentence-BERT

- 采用双塔句向量模型,融合文本与视觉特征比对链接源与目标语义
- 在6万+数据集上召回率达96%,接近大模型性能
- 适合大规模网页质量检测,兼顾效率与隐私安全
Web应用依赖超链接连接信息资源,但网络动态性导致链接失效(link rot)和语义漂移(semantic drift)——即页面可访问但内容已不符上下文。传统工具仅检查HTTP状态码,无法发现语义不一致;而大型语言模型虽具语义理解能力,却存在延迟高、成本大、隐私风险等问题。本文提出SemLink,一种基于Siamese Sentence-BERT架构的自动化测试断言系统,通过计算锚文本、周边DOM元素及视觉特征与目标页面内容之间的语义一致性,实现语义级链接验证。为训练与评估,我们构建了包含超过60,000个语义正样本对的HWPP数据集。实验表明,SemLink在召回率上达到96.00%,媲美先进大模型(如GPT-5.2),同时运行速度提升约47.5倍,资源消耗显著降低。该工作弥合了传统语法检查器与昂贵生成式AI之间的差距,为自动化网页质量保障提供了高效可靠的解决方案。
原文摘要 · Abstract (English)
Web applications rely heavily on hyperlinks to connect disparate information resources. However, the dynamic nature of the web leads to link rot, where targets become unavailable, and more insidiously, semantic drift, where a valid HTTP 200 connection exists, but the target content no longer aligns with the source context. Traditional verification tools, which primarily function as crash oracles by checking HTTP status codes, often fail to detect semantic inconsistencies, thereby compromising web integrity and user experience. While Large Language Models (LLMs) offer semantic understanding, they suffer from high latency, privacy concerns, and prohibitive costs for large-scale regression testing. In this paper, we propose SemLink, a novel automated test oracle for semantic hyperlink verification. SemLink leverages a Siamese Neural Network architecture powered by a pre-trained Sentence-BERT (SBERT) backbone to compute the semantic coherence between a hyperlink's source context (anchor text, surrounding DOM elements, and visual features) and its target page content. To train and evaluate our model, we introduce the Hyperlink-Webpage Positive Pairs (HWPPs) dataset, a rigorously constructed corpus of over 60,000 semantic pairs. Our evaluation demonstrates that SemLink achieves a Recall of 96.00%, comparable to state-of-the-art LLMs (GPT-5.2), while operating approximately 47.5 times faster and requiring significantly fewer computational resources. This work bridges the gap between traditional syntactic checkers and expensive generative AI, offering a robust and efficient solution for automated web quality assurance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。