A semi-supervised log anomaly detection method based on active learning and self-training

Rui Wang; Zhidong Wu; Yaxi Li; Feng Li; Mingyang Zhang; Jianyi Liu

doi:10.1117/12.3026181

8 April 2024 A semi-supervised log anomaly detection method based on active learning and self-training

Rui Wang, Zhidong Wu, Yaxi Li, Feng Li, Mingyang Zhang, Jianyi Liu

Proceedings Volume 13090, International Conference on Computer Application and Information Security (ICCAIS 2023); 130902M (2024) https://doi.org/10.1117/12.3026181
Event: International Conference on Computer Application and Information Security (ICCAIS 2023), 2023, Wuhan, China

Abstract

To address the issue of poor performance of anomaly detection models when the number of labelled logs is small, this paper proposes a semi supervised log anomaly detection method LogALST based on active learning and self-training. Self-training predicts a large number of unlabelled logs to generate high confidence log data, reducing the manual labelling cost of active learning. Active learning uses a sample sampling strategy to select log samples with high uncertainty, reducing labelling errors in unlabelled logs during the self-training prediction stage. Through a series of experimental verification, LogALST outperforms existing semi supervised log anomaly detection methods. At the same time, LogALST is also superior to log anomaly detection methods that rely solely on active learning or self-training, achieving better anomaly detection performance with less manual labelling costs.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Rui Wang, Zhidong Wu, Yaxi Li, Feng Li, Mingyang Zhang, and Jianyi Liu "A semi-supervised log anomaly detection method based on active learning and self-training", Proc. SPIE 13090, International Conference on Computer Application and Information Security (ICCAIS 2023), 130902M (8 April 2024); https://doi.org/10.1117/12.3026181

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available