Showing papers on "Weighted Majority Algorithm published in 2018"
TL;DR: Experimental results and analysis revealed that ABC algorithm perform best for majority of the problems on high dimension, while on small dimension, CS is the best choice, and FPA attain the next best position follow by BA and FA for all kinds of functions.
Abstract: In the past few years nature-inspired algorithms are seen as potential tools to solve computationally hard problems. Tremendous success of these algorithms in providing near optimal solutions has inspired the researchers to develop new algorithms. However, very limited efforts have been made to identify the best algorithms for diverse classes of problems. This work attempts to assess the efficacy of five contemporary nature-inspired algorithms i.e. bat algorithm (BA), artificial bee colony algorithm (ABC), cuckoo search algorithm (CS), firefly algorithm (FA) and flower pollination algorithm (FPA). The work evaluates the performance of these algorithms on CEC2014 30 benchmark functions which include unimodal, multimodal, hybrid and composite problems over 10, 30, 50 and 100 dimensions. Control parameters of all algorithms are self-adapted so as to obtain best results over benchmark functions. The algorithms have been evaluated along three perspectives (a) statistical significance using Wilcoxon rank sum test (b) computational time complexity (c) convergence rate of algorithms. Experimental results and analysis revealed that ABC algorithm perform best for majority of the problems on high dimension, while on small dimension, CS is the best choice. FPA attain the next best position follow by BA and FA for all kinds of functions. Self adaptation of above algorithms also revealed the best values of input parameters for various algorithms. This study may aid experts and scientists of computational intelligence to solve intricate optimization problems.
29 citations
TL;DR: The basic idea of this algorithm is that the descent directions of the cost function with respect to complex-valued parameters are calculated by limited-memory BFGS algorithm and the learning step is determined by Armijo line search method.
Abstract: In this paper, a new learning algorithm is proposed for complex-valued feedforward neural networks (CVFNNs). The basic idea of this algorithm is that the descent directions of the cost function with respect to complex-valued parameters are calculated by limited-memory BFGS algorithm and the learning step is determined by Armijo line search method. Since the approximation of Hessian matrix is calculated by utilizing the information of the latest several iterations, the memory efficiency is improved. To keep away from the saturated ranges of activation functions, some gain parameters are adjusted together with weights and biases. Compared with some existing learning algorithms for CVFNNs, the convergence speed is faster and a deeper minima of the cost function can be reached by the developed algorithm. In addition, the effects of initial values of weights and biases on the efficiency and convergence speed of the learning algorithm are analyzed. The performance of the proposed algorithm is evaluated in comparison with some existing classifiers on a variety of benchmark classification problems. Experimental results show that better performance is achieved by our algorithm with relatively compact network structure.
25 citations
Posted Content•
TL;DR: The presented framework opens more detailed questions about network topology; it is a bridge to the well studied techniques of semigroup theory and applying these techniques to answer what specific network topologies are capable of predicting.
Abstract: We show how well known rules of back propagation arise from a weighted combination of finite automata. By redefining a finite automata as a predictor we combine the set of all $k$-state finite automata using a weighted majority algorithm. This aggregated prediction algorithm can be simplified using symmetry, and we prove the equivalence of an algorithm that does this. We demonstrate that this algorithm is equivalent to a form of a back propagation acting in a completely connected $k$-node neural network. Thus the use of the weighted majority algorithm allows a bound on the general performance of deep learning approaches to prediction via known results from online statistics. The presented framework opens more detailed questions about network topology; it is a bridge to the well studied techniques of semigroup theory and applying these techniques to answer what specific network topologies are capable of predicting. This informs both the design of artificial networks and the exploration of neuroscience models.