用稀有关联规则检测交易数据中的异常,比传统方法更准。
RareSense: Rarity-Aware Similarity Search for Anomaly Retrieval in Transactional Data

- 通过挖掘最小稀有项集构建稀有规则,用加权余弦相似度比较对象
- 在四个基准数据集上,宏平均召回率领先,尤其对重复稀有结构的异常效果显著
- 适合需要发现罕见高阶模式的异常检测场景,如网络安全
对稀疏集合型数据进行相似性搜索时,频繁的背景属性常主导结果,因传统度量(如Jaccard、余弦、汉明)仅基于原子重叠。虽然逆文档频率(IDF)可缓解此问题,但仍是原子级且无法显式表达有意义的高阶共现。本文提出RareSense,一种面向稀疏交易型异常数据的稀有感知相似性框架。该方法挖掘最小稀有项集作为中间结构,生成可靠的稀有关联规则,将对象映射为稀疏稀有规则特征向量,并使用加权Jaccard相似度进行比较。规则权重结合逆支持度、置信度、提升度、结构复杂性和稳定性,使邻域由共享的稀有证据决定而非均匀特征重叠。我们证明,IDF加权的Jaccard是RareSense的一个受限单例,且诱导的距离在原始对象上为伪度量,在规则特征相同的等价类上为度量。在涵盖网络安全与一般分类领域的四个基准家族上实验显示,RareSense在查询条件下的宏平均检索性能最高,统计分析表明其显著优于原子基线。增益随工作负载变化,当异常共享可重复的稀有高阶结构时最为明显。在全局异常排序中,RareSense也达到最高宏平均表现,且与多个强检测器统计无显著差异。
原文摘要 · Abstract (English)
Similarity search over sparse set-valued data is often dominated by frequent background attributes because classical measures such as Jaccard, cosine, and Hamming compare objects through atomic overlap. IDF (Inverse document frequency) weighting partially reduces this effect but remains atom-wise and cannot explicitly represent informative higher-order co-occurrences. We introduce RareSense, a rarity-aware similarity framework for sparse transactional anomaly data. RareSense mines minimal rare itemsets as intermediate structures, derives reliable rare association rules, maps objects into sparse rare-rule profiles, and compares them using weighted Jaccard similarity. Rule weights combine inverse support, confidence, lift, structural complexity, and stability, so that neighborhoods are determined by shared rare evidence rather than uniform feature overlap. We show that IDF-weighted Jaccard is a restricted singleton case of RareSense, and that the induced distance is a pseudometric on the original objects and a metric over equivalence classes defined by identical rule profiles. Experiments across four benchmark families spanning cybersecurity and general categorical domains show that RareSense attains the highest observed macro-average query-conditioned retrieval performance among the evaluated similarity measures. The statistical analysis indicates significant overall differences, with corrected paired comparisons favoring RareSense over the atomic baselines. The gains remain workload-dependent and are strongest when anomalies share repeatable rare higher-order structure. For global anomaly ranking, RareSense achieves the highest observed macro-average performance while remaining statistically comparable to several strong dedicated detectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。