• 融合稀疏深度随机映射的隔离林异常检测方法

    Sparse deep random projection with isolation forest for anomaly detection

    • 孤立森林是一种高效的异常检测算法,但其轴平行线隔离的方式难以检测到非线性可分离数据中的异常。针对此问题,提出了一种利用神经网络随机投影融合孤立森林(isolation Forest, iForest)的半监督离群点检测方法——稀疏深度随机映射正向隔离林(Sparse deep Random Projection Positive isolation Forest, SRP-PiF)。算法通过神经网络进行非线性随机映射,将不同大小的子空间进行非线性划分,为分类提供有效的低维特征属性;设置超参数稀疏率来调整神经元中的权重,提高运算效率,最后结合半监督学习方式进行训练和预测。所提方法在真实世界10个数据集上进行验证,结果表明:相比于同类异常检测算法,SRP-PiF在异常检测性能指标AUC-ROC中有明显提升;在样本数量较多的数据集下,该算法识别精确度比孤立森林算法和经验累积分布函数的无监督离群值检测方法(ECOD)高出3% ~20%。

       

      Abstract: Isolation forest is an efficient anomaly detection algorithm, but it is difficult to detect anomalies in nonlinear separable data by the way of axial parallel line isolation. To solve this problem, a semi-supervised outliers detection method, Sparse deep Random Projection Positive isolation Forest (SRP-PiF), is proposed by using neural network random projection fusion isolation forest. Algorithm uses neural network to carry out nonlinear random projection, and subspaces of different sizes are divided nonlinearly to provide effective low-dimensional feature attributes for classification. The sparse rate of hyperparameter is set to adjust the weights in neurons and improve the efficiency of operation. Finally, the semi-supervised learning method is used for training and prediction. The method verified on 10 data sets in the real world. Compared with similar anomaly detection algorithms, SRP-PiF has a significant improvement in the anomaly detection performance index AUC-ROC. The accuracy of the algorithm is 3% - 20% higher than that of the isolation forest algorithm and the unsupervised outlier detection method of cumulative experience distribution function(ECOD) for large data sets.

       

    /

    返回文章
    返回