Web applications have become ubiquitous for many business sectors due to their platform independence and low operation cost. Billions of users are visiting these applications to accomplish their daily tasks. However, many of these applications are either vulnerable to web defacement attacks or created and managed by hackers such as fraudulent and phishing websites. Detecting malicious websites is essential to prevent the spreading of malware and protect end-users from being victims. However, most existing solutions rely on extracting features from the website’s content which can be harmful to the detection machines themselves and subject to obfuscations. Detecting malicious Uniform Resource Locators (URLs) is safer and more efficient than content analysis. However, the detection of malicious URLs is still not well addressed due to insufficient features and inaccurate classification. This study aims at improving the detection accuracy of malicious URL detection by designing and developing a cyber threat intelligence-based malicious URL detection model using two-stage ensemble learning. The cyber threat intelligence-based features are extracted from web searches to improve detection accuracy. Cybersecurity analysts and users reports around the globe can provide important information regarding malicious websites. Therefore, cyber threat intelligence-based (CTI) features extracted from Google searches and Whois websites are used to improve detection performance. The study also proposed a two-stage ensemble learning model that combines the random forest (RF) algorithm for preclassification with multilayer perceptron (MLP) for final decision making. The trained MLP classifier has replaced the majority voting scheme of the three trained random forest classifiers for decision making. The probabilistic output of the weak classifiers of the random forest was aggregated and used as input for the MLP classifier for adequate classification. Results show that the extracted CTI-based features with the two-stage classification outperform other studies’ detection models. The proposed CTI-based detection model achieved a 7.8% accuracy improvement and 6.7% reduction in false-positive rates compared with the traditional URL-based model.

/pdf/cyber-threat-intelligence-based-malicious-url-detection-sjc9pyxp.pdf

Cyber Threat Intelligence-Based Malicious URL Detection Model Using Ensemble Learning

In recent years, the digital world has advanced significantly, particularly on the Internet, which is critical given that many of our activities are now conducted online. As a result of attackers’ inventive techniques, the risk of a cyberattack is rising rapidly. One of the most critical attacks is the malicious URL intended to extract unsolicited information by mainly tricking inexperienced end users, resulting in compromising the user’s system and causing losses of billions of dollars each year. As a result, securing websites is becoming more critical. In this paper, we provide an extensive literature review highlighting the main techniques used to detect malicious URLs that are based on machine learning models, taking into consideration the limitations in the literature, detection technologies, feature types, and the datasets used. Moreover, due to the lack of studies related to malicious Arabic website detection, we highlight the directions of studies in this context. Finally, as a result of the analysis that we conducted on the selected studies, we present challenges that might degrade the quality of malicious URL detectors, along with possible solutions. 

/pdf/detecting-malicious-urls-using-machine-learning-techniques-2hhjtbet.pdf

Detecting Malicious URLs Using Machine Learning Techniques: Review and Research Directions

Business failure prediction is very important for the sustainable development of enterprises. Machine learning algorithms, especially ensemble algorithms, have shown great economic benefits in enterprise financial early warning. However, the highly imbalanced class distribution of financial risk data and the inexplainable of most machine learning-based early distress warning models limit their commercial application. To address the above limitations, we enhance the business failure prediction performance by tree-ensemble in a boosting manner. Moreover, to solve the class imbalanced issue in business failure datasets, a weighted objective function, weighted cross-entropy, is embedded into the boosted tree framework, making the weighted XGBoost a cost-sensitive business failure prediction model. Besides, to tackle the second issue, we explore the intrinsic interpretability of the proposed method by visualizing the feature importance and incorporating a partial dependence plot technique to locally interpret the individual business failure event. Experimental results on business failure datasets with different predictive horizons collected from China Security Market Accounting Research (CSMAR) database show the proposed weighted XGBoost is a good solution to reduce the error on recognizing firms in business failure. Furthermore, the visualized feature importance score and partial dependence plot result both demonstrate that the cost-sensitive tree-based ensemble can be a good tool to guide the investors in making rational as well as provide interpretable business prediction results as a reference for the policy-making of the regulators. 

/pdf/business-failure-prediction-based-on-a-cost-sensitive-2leoz2kr.pdf

Business Failure Prediction Based on a Cost-Sensitive Extreme Gradient Boosting Machine

Business failure prediction is very important for the sustainable development of enterprises. Machine learning algorithms, especially ensemble algorithms, have shown great economic benefits in enterprise financial early warning. However, the highly imbalanced class distribution of financial risk data and the inexplainable of most machine learning-based early distress warning models limit their commercial application. To address the above limitations, we enhance the business failure prediction performance by tree-ensemble in a boosting manner. Moreover, to solve the class imbalanced issue in business failure datasets, a weighted objective function, weighted cross-entropy, is embedded into the boosted tree framework, making the weighted XGBoost a cost-sensitive business failure prediction model. Besides, to tackle the second issue, we explore the intrinsic interpretability of the proposed method by visualizing the feature importance and incorporating a partial dependence plot technique to locally interpret the individual business failure event. Experimental results on business failure datasets with different predictive horizons collected from China Security Market Accounting Research (CSMAR) database show the proposed weighted XGBoost is a good solution to reduce the error on recognizing firms in business failure. Furthermore, the visualized feature importance score and partial dependence plot result both demonstrate that the cost-sensitive tree-based ensemble can be a good tool to guide the investors in making rational as well as provide interpretable business prediction results as a reference for the policy-making of the regulators.

https://ieeexplore.ieee.org/ielx7/6287639/6514899/09760439.pdf

Web phishing is a form of cybercrime aimed at tricking people into visiting malicious URLs to exfiltrate sensitive data. Since the structure of a malicious URL evolves over time, phishing detection mechanisms that can adapt to such variations are paramount. Furthermore, web phishing detection is an unbalanced classification task, as legitimate URLs outnumber malicious ones in real-life cases. Deep learning (DL) has emerged as a promising technique to minimize concept drift to enhance web phishing detection. Deep reinforcement learning (DRL) combines DL with reinforcement learning (RL); that is, a sequential decision-making paradigm in which the problem to be addressed is expressed as a Markov decision process (MDP). Recent studies have proposed an ad hoc MDP formulation to tackle unbalanced classification tasks called the imbalanced classification Markov decision process (ICMDP). In this paper, we exploit the ICMDP to present a double deep Q-Network (DDQN)-based classifier to address the unbalanced web phishing classification problem. The proposed algorithm is evaluated on a Mendeley web phishing dataset, from which three different data imbalance scenarios are generated. Despite a significant training time, it results in better geometric mean, index of balanced accuracy, F1 score, and area under the ROC curve than other DL-based classifiers combined with data-level sampling techniques in all test cases.

Unbalanced Web Phishing Classification through Deep Reinforcement Learning

Imbalanced class has been a common problem encountered in the modeling process, and has attracted more and more attention from scholars. Biased classifiers, which limit the classifiers’ performance for minority classes, will be produced if the imbalanced ratio between the number of positive labels and negative labels is ignored. The synthetic minority over-sampling technique (SMOTE) is a very classic and popular over-sampling method, which is widely used to address this problem. However, SMOTE increases label noise and the training time during the over-sampling process. To improve the detection rate of minority classes while ensuring efficiency, we propose a cost-sensitive XGBoost (CS-XGB) for the imbalanced data problem. The CS-XGB method can reduce the classifiers’ preference for most classes without changing the distribution of the original data. 600000 Uniform Resource Locators (URLs) were collected to validate the CS-XGB method. We compare XGBoost (XGB), SMOTE+XGB and CS-XGB, and the experimental results confirm that the CS-XGB is robust and efficient for imbalanced cases.

/pdf/an-effective-cost-sensitive-xgboost-method-for-malicious-ojv0jiv3oh.pdf

An Effective Cost-Sensitive XGBoost Method for Malicious URLs Detection in Imbalanced Dataset

With the rapid development of the Internet, the number of malicious URLs is growing, too. It brings great security threats to people. The current malicious URL detection methods based on machine learning don't select the features effectively, so it will directly affect the classification effect of the model. In order to avoid the problem of weak generalization ability caused by the training of small amount of samples, a large number of samples are used for experiments. In this paper, Random Forest is optimized based on feature contribution and hyperparameter optimization, and a large number of sample experiments show that the detection efficiency of the model has been significantly improved. The detection precision reaches 94.85%, and the AUC value reaches 96.51%. It has high application value for malicious URL detection.

Research on Malicious URL Detection Based on Feature Contribution Tendency

Domain Name System (DNS) service is a core service on the Internet and a key link to ensure the normal operation of the Internet. Since DNS can often pass through firewalls without being intercepted, it creates favorable conditions for attackers to build a covert channel based on the DNS protocol. DNS over HTTPS (DoH) can encrypt DNS lookup and response data packets to ensure that data packets are not monitored and used, but it also makes the DNS tunnels more difficult to detect. Due to the security of DoH, researchers began trying to detect DoH tunnels by using machine or deep learning. The effect of the model is not good if the data quality is poor or the amount of data is insufficient. Due to the privacy of traffic data, it is usually difficult to collect and share private traffic data to a centralized server. We propose a federated-learning DoH traffic classification framework (FL_DoH_CF), which permits multiple institutions to detect DoH tunnels by using convolutional neural network (CNN) without sharing traffic data. The experiments demonstrate that FL_DoH_CF is competitive with centralized learning, and it is still robust for non-independent and identically distributed (No_IID) data, and even achieves an accuracy of 99.86% for extreme one-class No_IID data.

Detecting DoH tunnels with privacy protection using federated learning

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Jun Xin

Author Tools

Chat about Author

Papers

An Effective Cost-Sensitive XGBoost Method for Malicious URLs Detection in Imbalanced Dataset

Research on Malicious URL Detection Based on Feature Contribution Tendency

Detecting DoH tunnels with privacy protection using federated learning