Journal Article10.1016/J.KNOSYS.2016.06.026
A structure optimization framework for feed-forward neural networks using sparse representation
20
TL;DR: A sparse-representation based framework, termed SRS, is introduced to generate a small-sized network structure without compromising the network performance and Experimental results indicate that the SRS framework performs favourably compared to alternative structure optimization algorithms.
read more
Abstract: Traditionally, optimizing the structure of a feed-forward neural-network is time-consuming and it needs to balance the trade-off between the network size and network performance. In this paper, a sparse-representation based framework, termed SRS, is introduced to generate a small-sized network structure without compromising the network performance. Based on the forward selection strategy, the SRS framework selects significant elements (weights or hidden neurons) from the initial network that minimize the residual output error. The main advantage of the SRS framework is that it is able to optimize the network structure and training performance simultaneously. As a result, the training error is reduced while the number of selected elements increases. The efficiency and robustness of the SRS framework are evaluated based on several benchmark datasets. Experimental results indicate that the SRS framework performs favourably compared to alternative structure optimization algorithms.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A hybrid hourly natural gas demand forecasting method based on the integration of wavelet transform and enhanced Deep-RNN model
TL;DR: This work proposes a robust hybrid hours-ahead gas consumption method by integrating Wavelet Transform, RNN-structured deep learning and Genetic Algorithm, and the dropout technology is applied in this work to overcome the potential problem of overfitting.
123
An improved algorithm for building self-organizing feedforward neural networks
TL;DR: This paper proposes a hybrid constructing and pruning strategy (HCPS) for problem solving, where the mutual information and sensitivity analysis are employed to measure the amount of internal information of neurons at the hidden layer and the contribution rate of each hidden neuron, respectively.
27
Dictionary learning based on discriminative energy contribution for image classification
Wenjie Zhu,Yunhui Yan,Yishu Peng +2 more
TL;DR: This paper addresses the feature extraction via learning a dictionary, whose sub dictionaries preserve correspondence to the class labels, and an optimal linear classifier jointly based on the structure of energy contribution, and learns a classifier to make the dictionary optimal and have a low cost on classifying.
11
Extreme learning machine based on cross entropy
Yixin Cui,Junhai Zhai,Xizhao Wang +2 more
- 10 Jul 2016
TL;DR: The proposed Cross Entropy based ELM (CE-ELM) can sufficiently overcome the drawback of overfitting in ELM with many hidden layer nodes, in which the mean square error minimization principle is replaced with the cross entropy minimization Principle.
10
Determination of the weights of external conditions for ship resistance
TL;DR: In this paper , an artificial neural network model was developed and the weights of external environmental conditions on the ship resistance were calculated and the results reveal that the two most dominant conditions affecting resistance are swell direction (33%) and wave direction (28%).
7
References
Extreme learning machine: Theory and applications
TL;DR: A new learning algorithm called ELM is proposed for feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs which tends to provide good generalization performance at extremely fast learning speed.
11.6K
Matching pursuits with time-frequency dictionaries
Stéphane Mallat,Zhifeng Zhang +1 more
TL;DR: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions, chosen in order to best match the signal structures.
10.2K
Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit
Joel A. Tropp,Anna C. Gilbert +1 more
TL;DR: It is demonstrated theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal.
Signal Recovery from Random Measurements Via Orthogonal Matching Pursuit: The Gaussian Case
Joel A. Tropp,Anna C. Gilbert +1 more
- 01 Aug 2007
TL;DR: In this paper, a greedy algorithm called Orthogonal Matching Pursuit (OMP) was proposed to recover a signal with m nonzero entries in dimension 1 given O(m n d) random linear measurements of that signal.