Abstract: Supervised machine learning algorithms have been a dominant method in the data mining field. Disease prediction using health data has recently shown a potential application area for these methods. This study aims to identify the key trends among different types of supervised machine learning algorithms, and their performance and usage for disease risk prediction. In this study, extensive research efforts were made to identify those studies that applied more than one supervised machine learning algorithm on single disease prediction. Two databases (i.e., Scopus and PubMed) were searched for different types of search items. Thus, we selected 48 articles in total for the comparison among variants supervised machine learning algorithms for disease prediction. We found that the Support Vector Machine (SVM) algorithm is applied most frequently (in 29 studies) followed by the Naive Bayes algorithm (in 23 studies). However, the Random Forest (RF) algorithm showed superior accuracy comparatively. Of the 17 studies where it was applied, RF showed the highest accuracy in 9 of them, i.e., 53%. This was followed by SVM which topped in 41% of the studies it was considered. This study provides a wide overview of the relative performance of different variants of supervised machine learning algorithms for disease prediction. This important information of relative performance can be used to aid researchers in the selection of an appropriate supervised machine learning algorithm for their studies.

Abstract: Fault diagnosis plays an important role in modern industry. With the development of smart manufacturing, the data-driven fault diagnosis becomes hot. However, traditional methods have two shortcomings: 1) their performances depend on the good design of handcrafted features of data, but it is difficult to predesign these features and 2) they work well under a general assumption: the training data and testing data should be drawn from the same distribution, but this assumption fails in many engineering applications. Since deep learning (DL) can extract the hierarchical representation features of raw data, and transfer learning provides a good way to perform a learning task on the different but related distribution datasets, deep transfer learning (DTL) has been developed for fault diagnosis. In this paper, a new DTL method is proposed. It uses a three-layer sparse auto-encoder to extract the features of raw data, and applies the maximum mean discrepancy term to minimizing the discrepancy penalty between the features from training data and testing data. The proposed DTL is tested on the famous motor bearing dataset from the Case Western Reserve University. The results show a good improvement, and DTL achieves higher prediction accuracies on most experiments than DL. The prediction accuracy of DTL, which is as high as 99.82%, is better than the results of other algorithms, including deep belief network, sparse filter, artificial neural network, support vector machine and some other traditional methods. What is more, two additional analytical experiments are conducted. The results show that a good unlabeled third dataset may be helpful to DTL, and a good linear relationship between the final prediction accuracies and their standard deviations have been observed.

Abstract: Following visible successes on a wide range of predictive tasks, machine learning techniques are attracting substantial interest from medical researchers and clinicians. We address the need for capacity development in this area by providing a conceptual introduction to machine learning alongside a practical guide to developing and evaluating predictive algorithms using freely-available open source software and public domain data. We demonstrate the use of machine learning techniques by developing three predictive models for cancer diagnosis using descriptions of nuclei sampled from breast masses. These algorithms include regularized General Linear Model regression (GLMs), Support Vector Machines (SVMs) with a radial basis function kernel, and single-layer Artificial Neural Networks. The publicly-available dataset describing the breast mass samples (N=683) was randomly split into evaluation (n=456) and validation (n=227) samples. We trained algorithms on data from the evaluation sample before they were used to predict the diagnostic outcome in the validation dataset. We compared the predictions made on the validation datasets with the real-world diagnostic decisions to calculate the accuracy, sensitivity, and specificity of the three models. We explored the use of averaging and voting ensembles to improve predictive performance. We provide a step-by-step guide to developing algorithms using the open-source R statistical programming environment. The trained algorithms were able to classify cell nuclei with high accuracy (.94 -.96), sensitivity (.97 -.99), and specificity (.85 -.94). Maximum accuracy (.96) and area under the curve (.97) was achieved using the SVM algorithm. Prediction performance increased marginally (accuracy =.97, sensitivity =.99, specificity =.95) when algorithms were arranged into a voting ensemble. We use a straightforward example to demonstrate the theory and practice of machine learning for clinicians and medical researchers. The principals which we demonstrate here can be readily applied to other complex tasks including natural language processing and image recognition.

Abstract: Medical image classification plays an essential role in clinical treatment and teaching tasks. However, the traditional method has reached its ceiling on performance. Moreover, by using them, much time and effort need to be spent on extracting and selecting classification features. The deep neural network is an emerging machine learning method that has proven its potential for different classification tasks. Notably, the convolutional neural network dominates with the best results on varying image classification tasks. However, medical image datasets are hard to collect because it needs a lot of professional expertise to label them. Therefore, this paper researches how to apply the convolutional neural network (CNN) based algorithm on a chest X-ray dataset to classify pneumonia. Three techniques are evaluated through experiments. These are linear support vector machine classifier with local rotation and orientation free features, transfer learning on two convolutional neural network models: Visual Geometry Group i.e., VGG16 and InceptionV3, and a capsule network training from scratch. Data augmentation is a data preprocessing method applied to all three methods. The results of the experiments show that data augmentation generally is an effective way for all three algorithms to improve performance. Also, Transfer learning is a more useful classification method on a small dataset compared to a support vector machine with oriented fast and rotated binary (ORB) robust independent elementary features and capsule network. In transfer learning, retraining specific features on a new target dataset is essential to improve performance. And, the second important factor is a proper network complexity that matches the scale of the dataset.

Abstract: Attack and anomaly detection in the Internet of Things (IoT) infrastructure is a rising concern in the domain of IoT. With the increased use of IoT infrastructure in every domain, threats and attacks in these infrastructures are also growing commensurately. Denial of Service, Data Type Probing, Malicious Control, Malicious Operation, Scan, Spying and Wrong Setup are such attacks and anomalies which can cause an IoT system failure. In this paper, performances of several machine learning models have been compared to predict attacks and anomalies on the IoT systems accurately. The machine learning (ML) algorithms that have been used here are Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), and Artificial Neural Network (ANN). The evaluation metrics used in the comparison of performance are accuracy, precision, recall, f1 score, and area under the Receiver Operating Characteristic Curve. The system obtained 99.4% test accuracy for Decision Tree, Random Forest, and ANN. Though these techniques have the same accuracy, other metrics prove that Random Forest performs comparatively better.

Abstract: There is a growing demand for detailed and accurate landslide maps and inventories around the globe, but particularly in hazard-prone regions such as the Himalayas. Most standard mapping methods require expert knowledge, supervision and fieldwork. In this study, we use optical data from the Rapid Eye satellite and topographic factors to analyze the potential of machine learning methods, i.e., artificial neural network (ANN), support vector machines (SVM) and random forest (RF), and different deep-learning convolution neural networks (CNNs) for landslide detection. We use two training zones and one test zone to independently evaluate the performance of different methods in the highly landslide-prone Rasuwa district in Nepal. Twenty different maps are created using ANN, SVM and RF and different CNN instantiations and are compared against the results of extensive fieldwork through a mean intersection-over-union (mIOU) and other common metrics. This accuracy assessment yields the best result of 78.26% mIOU for a small window size CNN, which uses spectral information only. The additional information from a 5 m digital elevation model helps to discriminate between human settlements and landslides but does not improve the overall classification accuracy. CNNs do not automatically outperform ANN, SVM and RF, although this is sometimes claimed. Rather, the performance of CNNs strongly depends on their design, i.e., layer depth, input window sizes and training strategies. Here, we conclude that the CNN method is still in its infancy as most researchers will either use predefined parameters in solutions like Google TensorFlow or will apply different settings in a trial-and-error manner. Nevertheless, deep-learning can improve landslide mapping in the future if the effects of the different designs are better understood, enough training samples exist, and the effects of augmentation strategies to artificially increase the number of existing samples are better understood.

Abstract: k-Nearest Neighbor (kNN) algorithm is an effortless but productive machine learning algorithm. It is effective for classification as well as regression. However, it is more widely used for classification prediction. kNN groups the data into coherent clusters or subsets and classifies the newly inputted data based on its similarity with previously trained data. The input is assigned to the class with which it shares the most nearest neighbors. Though kNN is effective, it has many weaknesses. This paper highlights the kNN method and its modified versions available in previously done researches. These variants remove the weaknesses of kNN and provide a more efficient method.

Abstract: Support vector machine (SVM) is an optimal margin based classification technique in machine learning. SVM is a binary linear classifier which has been extended to non-linear data using Kernels and multi-class data using various techniques like one-versus-one, one-versus-rest, Crammer Singer SVM, Weston Watkins SVM and directed acyclic graph SVM (DAGSVM) etc. SVM with a linear Kernel is called linear SVM and one with a non-linear Kernel is called non-linear SVM. Linear SVM is an efficient technique for high dimensional data applications like document classification, word-sense disambiguation, drug design etc. because under such data applications, test accuracy of linear SVM is closer to non-linear SVM while its training is much faster than non-linear SVM. SVM is continuously evolving since its inception and researchers have proposed many problem formulations, solvers and strategies for solving SVM. Moreover, due to advancements in the technology, data has taken the form of ‘Big Data’ which have posed a challenge for Machine Learning to train a classifier on this large-scale data. In this paper, we have presented a review on evolution of linear support vector machine classification, its solvers, strategies to improve solvers, experimental results, current challenges and research directions.

Abstract: Aiming at the problem that the most existing fault diagnosis methods could not effectively recognize the early faults in the rotating machinery, the empirical mode decomposition, fuzzy information entropy, improved particle swarm optimization algorithm and least squares support vector machines are introduced into the fault diagnosis to propose a novel intelligent diagnosis method, which is applied to diagnose the faults of the motor bearing in this paper. In the proposed method, the vibration signal is decomposed into a set of intrinsic mode functions (IMFs) by using empirical mode decomposition method. The fuzzy information entropy values of IMFs are calculated to reveal the intrinsic characteristics of the vibration signal and considered as feature vectors. Then the diversity mutation strategy, neighborhood mutation strategy, learning factor strategy and inertia weight strategy for basic particle swarm optimization (PSO) algorithm are used to propose an improved PSO algorithm. The improved PSO algorithm is used to optimize the parameters of least squares support vector machines (LS-SVM) in order to construct an optimal LS-SVM classifier, which is used to classify the fault. Finally, the proposed fault diagnosis method is fully evaluated by experiments and comparative studies for motor bearing. The experiment results indicate that the fuzzy information entropy can accurately and more completely extract the characteristics of the vibration signal. The improved PSO algorithm can effectively improve the classification accuracy of LS-SVM, and the proposed fault diagnosis method outperforms the other mentioned methods in this paper and published in the literature. It provides a new method for fault diagnosis of rotating machinery.

Abstract: It is important to detect breast cancer as early as possible. In this manuscript, a new methodology for classifying breast cancer using deep learning and some segmentation techniques are introduced. A new computer aided detection (CAD) system is proposed for classifying benign and malignant mass tumors in breast mammography images. In this CAD system, two segmentation approaches are used. The first approach involves determining the region of interest (ROI) manually, while the second approach uses the technique of threshold and region based. The deep convolutional neural network (DCNN) is used for feature extraction. A well-known DCNN architecture named AlexNet is used and is fine-tuned to classify two classes instead of 1,000 classes. The last fully connected (fc) layer is connected to the support vector machine (SVM) classifier to obtain better accuracy. The results are obtained using the following publicly available datasets (1) the digital database for screening mammography (DDSM); and (2) the Curated Breast Imaging Subset of DDSM (CBIS-DDSM). Training on a large number of data gives high accuracy rate. Nevertheless, the biomedical datasets contain a relatively small number of samples due to limited patient volume. Accordingly, data augmentation is a method for increasing the size of the input data by generating new data from the original input data. There are many forms for the data augmentation; the one used here is the rotation. The accuracy of the new-trained DCNN architecture is 71.01% when cropping the ROI manually from the mammogram. The highest area under the curve (AUC) achieved was 0.88 (88%) for the samples obtained from both segmentation techniques. Moreover, when using the samples obtained from the CBIS-DDSM, the accuracy of the DCNN is increased to 73.6%. Consequently, the SVM accuracy becomes 87.2% with an AUC equaling to 0.94 (94%). This is the highest AUC value compared to previous work using the same conditions.

Abstract: Machine learning (ML) techniques have been widely used in many smart city sectors, where a huge amount of data is gathered from various (IoT) devices. As a typical ML model, support vector machine (SVM) enables efficient data classification and thereby finds its applications in real-world scenarios, such as disease diagnosis and anomaly detection. Training an SVM classifier usually requires a collection of labeled IoT data from multiple entities, raising great concerns about data privacy. Most of the existing solutions rely on an implicit assumption that the training data can be reliably collected from multiple data providers, which is often not the case in reality. To bridge the gap between ideal assumptions and realistic constraints, in this paper, we propose secureSVM , which is a privacy-preserving SVM training scheme over blockchain-based encrypted IoT data. We utilize the blockchain techniques to build a secure and reliable data sharing platform among multiple data providers, where IoT data is encrypted and then recorded on a distributed ledger. We design secure building blocks, such as secure polynomial multiplication and secure comparison, by employing a homomorphic cryptosystem, Paillier, and construct a secure SVM training algorithm, which requires only two interactions in a single iteration, with no need for a trusted third-party. Rigorous security analysis prove that the proposed scheme ensures the confidentiality of the sensitive data for each data provider as well as the SVM model parameters for data analysts. Extensive experiments demonstrates the efficiency of the proposed scheme.

Abstract: Defect prediction models—classifiers that identify defect-prone software modules—have configurable parameters that control their characteristics (e.g., the number of trees in a random forest). Recent studies show that these classifiers underperform when default settings are used. In this paper, we study the impact of automated parameter optimization on defect prediction models. Through a case study of 18 datasets, we find that automated parameter optimization: (1) improves AUC performance by up to 40 percentage points; (2) yields classifiers that are at least as stable as those trained using default settings; (3) substantially shifts the importance ranking of variables, with as few as 28 percent of the top-ranked variables in optimized classifiers also being top-ranked in non-optimized classifiers; (4) yields optimized settings for 17 of the 20 most sensitive parameters that transfer among datasets without a statistically significant drop in performance; and (5) adds less than 30 minutes of additional computation to 12 of the 26 studied classification techniques. While widely-used classification techniques like random forest and support vector machines are not optimization-sensitive, traditionally overlooked techniques like C5.0 and neural networks can actually outperform widely-used techniques after optimization is applied. This highlights the importance of exploring the parameter space when using parameter-sensitive classification techniques.

Abstract: Depression is viewed as the largest contributor to global disability and a major reason for suicide. It has an impact on the language usage reflected in the written text. The key objective of our study is to examine Reddit users' posts to detect any factors that may reveal the depression attitudes of relevant online users. For such purpose, we employ the Natural Language Processing (NLP) techniques and machine learning approaches to train the data and evaluate the efficiency of our proposed method. We identify a lexicon of terms that are more common among depressed accounts. The results show that our proposed method can significantly improve performance accuracy. The best single feature is bigram with the Support Vector Machine (SVM) classifier to detect depression with 80% accuracy and 0.80 F1 scores. The strength and effectiveness of the combined features (LIWC+LDA+bigram) are most successfully demonstrated with the Multilayer Perceptron (MLP) classifier resulting in the top performance for depression detection reaching 91% accuracy and 0.93 F1 scores. According to our study, better performance improvement can be achieved by proper feature selections and their multiple feature combinations.

Abstract: For the smart grid energy theft identification, this letter introduces a gradient boosting theft detector (GBTD) based on the three latest gradient boosting classifiers (GBCs): 1) extreme gradient boosting; 2) categorical boosting; and 3) light gradient boosting method. While most of existing machine learning (ML) algorithms just focus on fine tuning the hyperparameters of the classifiers, our ML algorithm, GBTD, focuses on the feature engineering-based preprocessing to improve detection performance as well as time-complexity. GBTD improves both detection rate and false positive rate (FPR) of those GBCs by generating stochastic features like standard deviation, mean, minimum, and maximum value of daily electricity usage. GBTD also reduces the classifier complexity with weighted feature-importance-based extraction techniques. Emphasis has been laid upon the practical application of the proposed ML for theft detection by minimizing FPR and reducing data storage space and improving time-complexity of the GBTD classifiers. Additionally, this letter proposes an updated version of the existing six theft cases to mimic real-world theft patterns and applies them to the dataset for numerical evaluation of the proposed algorithm.

Abstract: Above-ground biomass (AGB) is a basic agronomic parameter for field investigation and is frequently used to indicate crop growth status, the effects of agricultural management practices, and the ability to sequester carbon above and below ground. The conventional way to obtain AGB is to use destructive sampling methods that require manual harvesting of crops, weighing, and recording, which makes large-area, long-term measurements challenging and time consuming. However, with the diversity of platforms and sensors and the improvements in spatial and spectral resolution, remote sensing is now regarded as the best technical means for monitoring and estimating AGB over large areas. In this study, we used structural and spectral information provided by remote sensing from an unmanned aerial vehicle (UAV) in combination with machine learning to estimate maize biomass. Of the 14 predictor variables, six were selected to create a model by using a recursive feature elimination algorithm. Four machine-learning regression algorithms (multiple linear regression, support vector machine, artificial neural network, and random forest) were evaluated and compared to create a suitable model, following which we tested whether the two sampling methods influence the training model. To estimate the AGB of maize, we propose an improved method for extracting plant height from UAV images and a volumetric indicator (i.e., BIOVP). The results show that (1) the random forest model gave the most balanced results, with low error and a high ratio of the explained variance for both the training set and the test set. (2) BIOVP can retain the largest strength effect on the AGB estimate in four different machine learning models by using importance analysis of predictors. (3) Comparing the plant heights calculated by the three methods with manual ground-based measurements shows that the proposed method increased the ratio of the explained variance and reduced errors. These results lead us to conclude that the combination of machine learning with UAV remote sensing is a promising alternative for estimating AGB. This work suggests that structural and spectral information can be considered simultaneously rather than separately when estimating biophysical crop parameters.

Abstract: Support vector machines (SVMs) are a supervised classifier successfully applied in a plethora of real-life applications. However, they suffer from the important shortcomings of their high time and memory training complexities, which depend on the training set size. This issue is especially challenging nowadays, since the amount of data generated every second becomes tremendously large in many domains. This review provides an extensive survey on existing methods for selecting SVM training data from large datasets. We divide the state-of-the-art techniques into several categories. They help understand the underlying ideas behind these algorithms, which may be useful in designing new methods to deal with this important problem. The review is complemented with the discussion on the future research pathways which can make SVMs easier to exploit in practice.

Abstract: Diseases related to Heart i.e. Cardiovascular Dis-eases (CVDs) are the main reason for the number of deaths in the course of the most recent couple of decades and has developed as the most perilous ailment, in India and in the entire world. In this way, there is a need for accurate, feasible and reliable system to analyze such maladies in time for legitimate treatment. Machine Learning algorithms and procedures have been im-plemented to various medical datasets to various medical datasets to investigate of extensive and complex information. Numerous analysts, as of late, have been using several methods to enable the health care industry and the professionals in the diagnosis of heart related diseases. This paper demonstrates a survey of various models based on such algorithms and techniques and analyze their perfor-mance. Models depend on supervised learning algorithms such as Support Vector Machines (SVM), K-Nearest Neighbour (KNN), Naive Bayes, Decision Trees (DT), Random Forest (RF) and ensemble models are discovered extremely prominent among the researchers.

Abstract: Machine learning tools represent key enablers for empowering material scientists and engineers to accelerate the development of novel materials, processes and techniques. One of the aims of using such approaches in the field of materials science is to achieve high-throughput identification and quantification of essential features along the process-structure-property-performance chain. In this contribution, machine learning and statistical learning approaches are reviewed in terms of their successful application to specific problems in the field of continuum materials mechanics. They are categorized with respect to their type of task designated to be either descriptive, predictive or prescriptive; thus to ultimately achieve identification, prediction or even optimization of essential characteristics. The respective choice of the most appropriate machine learning approach highly depends on the specific use-case, type of material, kind of data involved, spatial and temporal scales, formats, and desired knowledge gain as well as affordable computational costs. Different examples are reviewed involving case-by-case dependent application of different types of artificial neural networks and other data-driven approaches such as support vector machines, decision trees and random forests as well as Bayesian learning, and model order reduction procedures such as principal component analysis, among others. These techniques are applied to accelerate the identification of material parameters or salient features for materials characterization, to support rapid design and optimization of novel materials or manufacturing methods, to improve and correct complex measurement devices, or to better understand and predict fatigue behavior, among other examples. Besides experimentally obtained datasets, numerous studies draw required information from simulation-based data mining. Altogether, it is shown that experiment- and simulation-based data mining in combination with machine leaning tools provide exceptional opportunities to enable highly reliant identification of fundamental interrelations within materials for characterization and optimization in a scale-bridging manner. Potentials of further utilizing applied machine learning in materials science and empowering significant acceleration of knowledge output are pointed out.

Abstract: Big multimedia data is heterogeneous in essence, that is, the data may be a mixture of video, audio, text, and images. This is due to the prevalence of novel applications in recent years, such as social media, video sharing, and location based services (LBS), etc. In many multimedia applications, for example, video/image tagging and multimedia recommendation, text classification techniques have been used extensively to facilitate multimedia data processing. In this paper, we give a comprehensive review on feature selection techniques for text classification. We begin by introducing some popular representation schemes for documents, and similarity measures used in text classification. Then, we review the most popular text classifiers, including Nearest Neighbor (NN) method, Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Neural Networks. Next, we survey four feature selection models, namely the filter, wrapper, embedded and hybrid, discussing pros and cons of the state-of-the-art feature selection approaches. Finally, we conclude the paper and give a brief introduction to some interesting feature selection work that does not belong to the four models.

Abstract: We present an approach that combines automatic features learned by convolutional neural networks (CNN) and handcrafted features computed by the bag-of-visual-words (BOVW) model in order to achieve the state-of-the-art results in facial expression recognition (FER). To obtain automatic features, we experiment with multiple CNN architectures, pre-trained models, and training procedures, e.g., Dense–Sparse–Dense. After fusing the two types of features, we employ a local learning framework to predict the class label for each test image. The local learning framework is based on three steps. First, a k-nearest neighbors model is applied in order to select the nearest training samples for an input test image. Second, a one-versus-all support vector machines (SVM) classifier is trained on the selected training samples. Finally, the SVM classifier is used to predict the class label only for the test image it was trained for. Although we have used local learning in combination with handcrafted features in our previous work, to the best of our knowledge, local learning has never been employed in combination with deep features. The experiments on the 2013 FER Challenge data set, the FER+ data set, and the AffectNet data set demonstrate that our approach achieves the state-of-the-art results. With a top accuracy of 75.42% on the FER 2013, 87.76% on the FER+, 59.58% on the AffectNet eight-way classification, and 63.31% on the AffectNet seven-way classification, we surpass the state-of-the-art methods by more than 1% on all data sets.

Abstract: Chest X-ray film is the most widely used and common method of clinical examination for pulmonary nodules. However, the number of radiologists obviously cannot keep up with this outburst due to the sharp increase in the number of pulmonary diseases, which increases the rate of missed diagnosis and misdiagnosis. The method based on deep learning is the most appropriate way to deal with such problems so far. The main research in this paper was using inception-v3 transfer learning model to classify pulmonary images, and finally to get a practical and feasible computer-aided diagnostic model. The computer-aided diagnostic model could improve the accuracy and rapidity of doctors in the diagnosis of thoracic diseases. In this experiment, we augmented the data of pulmonary images, then used the fine-tuned Inception-v3 model based on transfer learning to extract features automatically, and used different classifiers (Softmax, Logistic, SVM) to classify the pulmonary images. Finally, it was compared with various models based on the original Deep Convolution Neural Network (DCNN) model. The experiment proved that the experiment based on transfer learning was meaningful for pulmonary image classification. The highest sensitivity and specificity are 95.41% and 80.09% respectively in the experiment, and the better pulmonary image classification performance was obtained than other methods.