Journal Article10.1007/S11042-020-09054-7
Revisiting spatial dropout for regularizing convolutional neural networks
Lee Sang Hun,Chulhee Lee +1 more
49
TL;DR: It is found that dropout between channels in the CNNs can be functionally similar to dropout in the FCNNs, and spatial dropout can be an effective way to take advantage of the dropout technique for regularizing.
read more
Abstract: Overfitting is one of the most challenging problems in deep neural networks with a large number of trainable parameters To prevent networks from overfitting, the dropout method, which is a strong regularization technique, has been widely used in fully-connected neural networks In several state-of-the-art convolutional neural network architectures for object classification, however, dropout was partially or not even applied since its accuracy gain was relatively insignificant in most cases Also, the batch normalization technique reduced the need for the dropout method because of its regularization effect In this paper, we show that conventional element-wise dropout can be ineffective for convolutional layers We found that dropout between channels in the CNNs can be functionally similar to dropout in the FCNNs, and spatial dropout can be an effective way to take advantage of the dropout technique for regularizing To prove our points, we conducted several experiments using the CIFAR-10 and CIFAR-100 databases For comparison, we only replaced the dropout layers with spatial dropout layers and kept all other hyperparameters and methods intact DenseNet-BC with spatial dropout showed promising results (332% error rates with CIFAR-10, 30 M parameters) compared to other existing competitive methods
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Reconstructing Undersampled Photoacoustic Microscopy Images Using Deep Learning
Anthony DiSpirito,Daiwei Li,Tri Vu,Maomao Chen,Dong Zhang,Jianwen Luo,Roarke Horstmeyer,Junjie Yao +7 more
TL;DR: The results and numerical analysis have collectively demonstrated the robust performance of the model to reconstruct PAM images with as few as 2% of the original pixels, which can effectively shorten the imaging time without substantially sacrificing the image quality.
136
Automated accurate emotion recognition system using rhythm-specific deep convolutional neural network technique with multi-channel EEG signals
TL;DR: In this article, the authors proposed a rhythm-specific multi-channel convolutional neural network (CNN) based approach for automated emotion recognition using multichannel electroencephalogram (EEG) signals.
96
Image Augmentation for Deep Learning Based Lesion Classification from Skin Images
Evgin Goceri
- 09 Dec 2020
TL;DR: Zhang et al. as discussed by the authors used image augmentation to solve the overfitting problem and obtain well-generalizing network models in deep learning-based skin lesion classification, which is an efficient approach to deal with this issue using existing images more efficiently.
57
AlexNet architecture based convolutional neural network for toxic comments classification
TL;DR: In this paper , a 3-tier CNN architecture was proposed to classify the toxic comments on the Wikipedia forum available in the Google Jigsaw dataset, which achieved a decent average accuracy of 98.505% and an average F1 score of 0.79.
42
Full-view in vivo skin and blood vessels profile segmentation in photoacoustic imaging based on deep learning.
Cao Duong Ly,Van Tu Nguyen,Tan Hung Vo,Sudip Mondal,Sumin Park,Jaeyeop Choi,Thi Thu Ha Vu,Chang-Seok Kim,Junghwan Oh +8 more
TL;DR: Wang et al. as discussed by the authors proposed a DL model based on modified U-Net for extracting the relationship features between amplitudes of the generated photoacoustic signal from skin and underlying vessels.
24
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
•Posted Content
Deep Residual Learning for Image Recognition
TL;DR: This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
117.9K
•Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
- 04 Sep 2014
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
102.6K
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
ImageNet: A large-scale hierarchical image database
Jia Deng,Wei Dong,Richard Socher,Li-Jia Li,Kai Li,Li Fei-Fei +5 more
- 20 Jun 2009
TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.