Open AccessJournal Article
Continuous Dropout
26
TL;DR: In this article, the authors extend the traditional binary dropout to continuous dropout, which is closer to the activation characteristics of neurons in the human brain than traditional binary Dropout.
read more
Abstract: Dropout has been proven to be an effective algorithm for training robust deep networks because of its ability to prevent overfitting by avoiding the co-adaptation of feature detectors Current explanations of dropout include bagging, naive Bayes, regularization, and sex in evolution According to the activation patterns of neurons in the human brain, when faced with different situations, the firing rates of neurons are random and continuous, not binary as current dropout does Inspired by this phenomenon, we extend the traditional binary dropout to continuous dropout On the one hand, continuous dropout is considerably closer to the activation characteristics of neurons in the human brain than traditional binary dropout On the other hand, we demonstrate that continuous dropout has the property of avoiding the co-adaptation of feature detectors, which suggests that we can extract more independent feature detectors for model averaging in the test stage We introduce the proposed continuous dropout to a feedforward neural network and comprehensively compare it with binary dropout, adaptive dropout, and DropConnect on Modified National Institute of Standards and Technology, Canadian Institute for Advanced Research-10, Street View House Numbers, NORB, and ImageNet large scale visual recognition competition-12 Thorough experiments demonstrate that our method performs better in preventing the co-adaptation of feature detectors and improves test performance
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Beyond Bilinear: Generalized Multimodal Factorized High-Order Pooling for Visual Question Answering
TL;DR: Zhang et al. as mentioned in this paper proposed a coattention mechanism using a deep neural network (DNN) architecture to jointly learn the attentions for both the image and the question, which can reduce the irrelevant features effectively and obtain more discriminative features for image and question representations.
Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization.
TL;DR: The proposed advanced dropout technique applies a model-free and easily implemented distribution with parametric prior, and adaptively adjusts dropout rate to carry out an end-to-end training of DNNs.
58
Robust manifold broad learning system for large-scale noisy chaotic time series prediction: A perturbation perspective.
TL;DR: The robust manifold broad learning system (RM-BLS) is proposed for system modeling and large-scale noisy chaotic time series prediction and the equivalence between perturbations to manifold embedding and Tikhonov regularization is proved.
57
Concrete Cracks Detection and Monitoring Using Deep Learning-Based Multiresolution Analysis
TL;DR: The ultimate objective of this work is to propose an automatic crack type identification scheme based on convolutional neural networks (CNN), which can be monitored without access to the concrete surface and the goal is to detect cracks before they are visible.
56
Optimization of a Deep-Learning Method Based on the Classification of Images Generated by Parameterized Deep Snap a Novel Molecular-Image-Input Technique for Quantitative Structure-Activity Relationship (QSAR) Analysis.
TL;DR: The effects of the loss in validation as an indicator for evaluating the performance of the DL using the toxicity information in the Tox21 qHTP database suggest that optimal thresholds exist to attain the best performance with these prediction models.
References
•Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
- 04 Sep 2014
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
102.6K
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
- 01 Jan 1998
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
53.5K
•Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky,Jia Deng,Hao Su,Jonathan Krause,Sanjeev Satheesh,Sean Ma,Zhiheng Huang,Andrej Karpathy,Aditya Khosla,Michael S. Bernstein,Alexander C. Berg,Li Fei-Fei +11 more
TL;DR: The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) as mentioned in this paper is a benchmark in object category classification and detection on hundreds of object categories and millions of images, which has been run annually from 2010 to present, attracting participation from more than fifty institutions.