Unique Log Parsing Framework for Enhanced Anomaly Detection in Network Security: LaukiLogParser | [International Journal of Communication Networks and Information Security • 2024]

Author(s):

1. Mukesh Yadav: Department of Computer Engineering SVKM’s NMIMS Deemed to be University Mukesh Patel School of Technology Management & Engineering Mumbai, India

2. Dhirendra S Mishra: Department of Computer Engineering, SVKM’s NMIMS Deemed to be University, Mukesh Patel School of Technology Management & Engineering, Muumbai, India

Abstract:

The increasing complexity of information security demands effective strategies to protect data across various domains. Traditional system log analysis, relying on unstructured logs, employs data mining and machine learning techniques to detect network threats. However, existing methods often struggle with logs of diverse formats and structures, resulting in missed anomalies and vulnerabilities. This paper introduces LaukiLogParser, a novel real-time log parsing framework designed to address these challenges by processing both structured and unstructured logs from multiple formats, including JSON, Syslog, and CEF. By incorporating unique parsing equations, the proposed parser enhances the identification of network threats, insider threats, and system vulnerabilities. Through comprehensive testing on publicly available datasets, LaukiLogParser demonstrated a significant 15% increase in anomaly detection accuracy compared to traditional parsers, along with improved F1-scores, precision, and recall. The parser's ability to handle a variety of log formats provides unmatched flexibility in real-time environments, making it highly effective for modern network security systems. The paper compares LaukiLogParser with existing parsers, such as LogParser-LLM, OpenLog, and LogPPT, showcasing its superiority in accuracy, scalability, and adaptability. The results highlight the limitations of current parsers, while LaukiLogParser's novel approach offers a robust solution for enhancing anomaly detection and improving real-time security monitoring.

Page(s): 890-905

DOI: DOI not available

Published: Journal: International Journal of Communication Networks and Information Security, Volume: 16, Issue: 4, Year: 2024

Keywords:

Cyber Security , Machine learning , Realtime Log Parsing , Cybersecurity Threat Identification , Multi format Log Analysis

References:

[1] Zhong A.,Mo D. .2024 ."LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models,". Available: DOI: 10.48550/arXiv.2408.13727, : .

[2] Ma Z. .2024 ."OpenLogParser: Unsupervised Parsing with Open-Source Large Language Models,". Available: DOI: 10.48550/arXiv.2408.01585, 59(1) : 101-120.

[3] .2023 .[3] Le and Zhang, "Semantic-based Log Parsing and Enhancements with LLMs,". Available: DOI: 10.48550/arXiv.2406.06156, : .

[4] Zhou Y. .2024 ."Stronger, Cheaper and Demonstration-Free Log Parsing with LLMs,". Available: DOI: 10.48550/arXiv.2406.06156, 31(2) : 539-555.

[5] Xu .2024 ."Variable-Aware Log Parsing with Pointer Network,". Available: DOI: 10.48550/arXiv.2401.05986, 21(3) : 300-314.

[6] Yu S. .2023 ."Log3T: Log Parsing with Generalization Ability under New Log Types,". in Proceedings of the ESEC/FSE, : .

[7] Zhang Y. .2023 ."AdaptParse: Adaptive Contextual Aware Attention Network for Log Parsing,". in IEEE Symposium on Security and Privacy, : .

[8] Patel L. K.,Rajpoot D. S.,Feb D. S. .2023 ."Advances in Log Parsing Techniques: A Survey,". Computers & Security, 10 : 102560.

Citations

Downloads

Views