arXiv:2606.20868cs.CRcs.AI2026-06

评测大模型识别品牌域名归属的能力,发现加外部工具可大幅降低误判风险。

Can LLMs Reason About Brand Ownership? An Empirical Study of Domain Attribution Intelligence

论文配图:Can LLMs Reason About Brand Ownership? An Empirical Study of Domain Attribution Intelligence
图 1 · 摘自论文原文
  • 用大模型结合查询工具判断域名是否属品牌所有
  • 仅靠记忆时准确率最高82%,无工具时分类性能极差
  • 使用WHOIS数据后误报率显著下降,适合安全防护场景

当出现与知名品牌相似的新域名时,防御方面临根本性困惑:该域名可能是攻击者注册的钓鱼网站,也可能是品牌方为防御或新产品注册的合法域名。错误将品牌自有域名标记为恶意会引发误报,损害用户体验和品牌声誉。解决此问题需具备品牌智能——大规模判断域名归属的能力。大语言模型(LLMs)凭借对品牌域名关系的广泛知识,提供了一种零配置解决方案,但其在品牌智能任务中的可靠性尚不明确。本文首次系统评估了四个模型(Gemini 2.5 Flash、Gemini 3.5 Flash、Claude Sonnet 4.5、Claude Sonnet 4.6)在三个任务上的表现:域名枚举(Q1)、开放式品牌归属推断(Q2)和二元所有权分类(Q3)。实验覆盖36个最常被钓鱼的品牌,在四种检索设置(上下文学习、网络搜索、WHOIS查询、组合)下进行。结果揭示显著差异:仅依赖记忆时,模型在域名枚举上最高达82%精度;但在无外部工具情况下,所有权验证性能极低,ICL模式下宏F1最高仅0.37。引入WHOIS信息使Q3任务宏F1提升最多0.65点,精确率达到≤0.99,显著降低防御方误报风险。研究为大模型在品牌保护流程中的部署提供了具体建议。

原文摘要 · Abstract (English)

When a new domain resembling a popular brand appears, defenders face a fundamental ambiguity: it may be an attacker-created squatting site for phishing, or it may be a domain the brand itself registered, either defensively, to block attackers, or legitimately, for a new product or service launch. Incorrectly flagging a brand-owned domain as malicious produces a false positive that harms end users and damages the brand's reputation. Resolving this ambiguity requires brand intelligence: the ability to determine, at scale, whether a given domain belongs to a brand. Large language models (LLMs), with their broad knowledge of brand domain relationships, offer a promising zero configuration approach to this problem, but their reliability for brand intelligence tasks remains unknown. We present the first systematic empirical evaluation of LLM brand intelligence across three tasks: domain enumeration (Q1), open ended brand attribution (Q2), and binary ownership classification (Q3). We evaluate four models, Gemini 2.5 Flash, Gemini 3.5 Flash, Claude Sonnet 4.5, and Claude Sonnet 4.6, across four retrieval settings (in context, web search, WHOIS lookup, and combined) on 36 of the most phished brands. Our results reveal a stark dichotomy: models achieve up to 82% precision enumerating brand domains from memory alone, yet fail at ownership verification without external tools, with macro F1 at most 0.37 in ICL mode. WHOIS augmentation lifts Q3 macro F1 by up to 0.65 points, yielding near perfect precision (<= 0.99), dramatically reducing the false positive risk for defenders. We provide concrete recommendations for deploying LLMs in brand protection pipelines.

大模型品牌安全域名检测零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。