Efficient Distributed Preprocessing Model for Machine Learning-Based Anomaly Detection over Large-Scale Cybersecurity Datasets
Xavier Larriva-Novo,Mario Vega-Barbas,Víctor A. Villagrá,Diego Rivera,Manuel Alvarez-Campana,Julio Berrocal +5 more
TL;DR: A new model of data preprocessing based on a novel distributed computing architecture focused on large-scale datasets such as UGR’16 is presented and the adequateness of decision tree algorithms for training a machine learning model is shown by using a large dataset when compared with a multilayer perceptron neural network.
read more
Abstract: New computational and technological paradigms that currently guide developments in the information society, i.e., Internet of things, pervasive technology, or Ubicomp, favor the appearance of new intrusion vectors that can directly affect people’s daily lives. This, together with advances in techniques and methods used for developing new cyber-attacks, exponentially increases the number of cyber threats which affect the information society. Because of this, the development and improvement of technology that assists cybersecurity experts to prevent and detect attacks arose as a fundamental pillar in the field of cybersecurity. Specifically, intrusion detection systems are now a fundamental tool in the provision of services through the internet. However, these systems have certain limitations, i.e., false positives, real-time analytics, etc., which require their operation to be supervised. Therefore, it is necessary to offer architectures and systems that favor an efficient analysis of the data handled by these tools. In this sense, this paper presents a new model of data preprocessing based on a novel distributed computing architecture focused on large-scale datasets such as UGR’16. In addition, the paper analyzes the use of machine learning techniques in order to improve the response and efficiency of the proposed preprocessing model. Thus, the solution developed achieves good results in terms of computer performance. Finally, the proposal shows the adequateness of decision tree algorithms for training a machine learning model by using a large dataset when compared with a multilayer perceptron neural network.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Analysis of Cyber Security Attacks and Its Solutions for the Smart grid Using Machine Learning and Blockchain Methods
TL;DR: In this paper , the authors look at the many risks and flaws that can affect the safety of critical, innovative grid network components, and propose security solutions using different methods, and also provide recommendations for reducing the chance that these three categories of cyberattacks may occur.
78
An IoT-Focused Intrusion Detection System Approach Based on Preprocessing Characterization for Cybersecurity Datasets.
TL;DR: In this paper, the authors proposed the study and evaluation of several preprocessing techniques based on traffic categorization for a machine learning neural network algorithm for intrusion detection in IoT networks, and evaluated these preprocessing models in accordance with scalar and normalization functions.
70
An Agile Approach to Identify Single and Hybrid Normalization for Enhancing Machine Learning-Based Network Intrusion Detection
TL;DR: In this article, a statistical method is proposed that can identify the most suitable normalization method for the dataset, which gives the highest accuracy for an intrusion detection system, and the proposed method is also able to identify hybrid normalizations to achieve even improved intrusion detection results.
Edge Intelligence in Smart Grids: A Survey on Architectures, Offloading Models, Cyber Security Measures, and Challenges
TL;DR: It is concluded that most of the viable architectures for EI in smart grids often consist of three layers: device, edge, and cloud, and it is crucial that computation offloading techniques must be framed as optimization problems and addressed effectively in order to increase system performance.
Prepare for trouble and make it double! Supervised – Unsupervised stacking for anomaly-based intrusion detection
Tommaso Zoppi,Andrea Ceccarelli +1 more
TL;DR: In this paper, a two-layer Stacker is proposed to detect unknown zero-day attacks by combining supervised and unsupervised algorithms, which is more effective in detecting unknown attacks than supervised algorithms.
22
References
Machine learning algorithms for accurate flow-based network traffic classification: Evaluation and comparison
Murat Soysal,Ece Guran Schmidt +1 more
TL;DR: The dependency of the traffic classification performance on the amount and composition of training data is investigated followed by experiments that show that ML algorithms such as Bayesian Networks and Decision Trees are suitable for Internet traffic flow classification at a high speed, and prove to be robust with respect to applications that dynamically change their source ports.
190
A Stacking Ensemble for Network Intrusion Detection Using Heterogeneous Datasets
TL;DR: An ensemble model using metaclassification approach enabled by stacked generalization is presented capable of generating superior predictions with respect to a real-time dataset than an emulated one.
Intrusion Detection System Based on Decision Tree over Big Data in Fog Environment
Kai Peng,Victor C. M. Leung,Lixin Zheng,Shangguang Wang,Chao Huang,Tao Lin +5 more
- 01 Mar 2018
TL;DR: This study proposes an IDS system based on decision tree and proposes a preprocessing algorithm to digitize the strings in the given dataset and then normalize the whole data to ensure the quality of the input data so as to improve the efficiency of detection.
A Framework for Fast and Efficient Cyber Security Network Intrusion Detection Using Apache Spark
Govind P. Gupta,Manish Kulariya +1 more
TL;DR: This paper has proposed a framework in which first a well-known feature selection algorithm is employed for selecting the most important features and then classification based intrusion detection method is used for fast and efficient detection of intrusion in the massive network traffic.
121
A novel architecture combined with optimal parameters for back propagation neural networks applied to anomaly network intrusion detection
TL;DR: This paper has proposed an optimal approach to build an effective anomaly NIDS based on Back Propagation Neural Network (BPNN) using Backpropagation Learning Algorithm, and employed a novel architecture for that network.
87