Auto-Encoders in Deep Learning—A Review with New Perspectives
Shuangshuang Chen,Wei Guo +1 more
TL;DR: The auto-encoder is a key component of deep structure, which can be used to realize transfer learning and plays an important role in both unsupervised learning and non-linear feature extraction as mentioned in this paper .
read more
Abstract: Deep learning, which is a subfield of machine learning, has opened a new era for the development of neural networks. The auto-encoder is a key component of deep structure, which can be used to realize transfer learning and plays an important role in both unsupervised learning and non-linear feature extraction. By highlighting the contributions and challenges of recent research papers, this work aims to review state-of-the-art auto-encoder algorithms. Firstly, we introduce the basic auto-encoder as well as its basic concept and structure. Secondly, we present a comprehensive summarization of different variants of the auto-encoder. Thirdly, we analyze and study auto-encoders from three different perspectives. We also discuss the relationships between auto-encoders, shallow models and other deep learning models. The auto-encoder and its variants have successfully been applied in a wide range of fields, such as pattern recognition, computer vision, data generation, recommender systems, etc. Then, we focus on the available toolkits for auto-encoders. Finally, this paper summarizes the future trends and challenges in designing and training auto-encoders. We hope that this survey will provide a good reference when using and designing AE models.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Autoencoders and their applications in machine learning: a survey
Kamal Berahmand,Fatemeh Daneshfar,Elaheh Sadat Salehi,Yuefeng Li,Yue Xu +4 more
TL;DR: A comprehensive survey of autoencoders is presented, starting with an explanation of the principle of conventional autoencoder and their primary development process, and a taxonomy of autoencoders based on their structures and principles is provided.
116
A comprehensive review of synthetic data generation in smart farming by using variational autoencoder and generative adversarial network
Yaganteeswarudu Akkem,Saroj Biswas,Aruna Varanasi +2 more
TL;DR: This study reviews the application of Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) for generating synthetic crop data, enhancing crop recommendation models with high-quality, realistic data, and improving their performance in agriculture.
49
Deep learning in computational mechanics: a review
Leon Herrmann,Stefan Kollmannsberger +1 more
TL;DR: This review focuses on deep learning methods rather than applications for computational mechanics, thereby enabling researchers to explore this field more effectively and help researchers identify key concepts and promising methodologies within this field.
42
Machine Learning Empowering Personalized Medicine: A Comprehensive Review of Medical Image Analysis Methods
Irena Galić,Marija Habijan,Hrvoje Leventic,Kresimir Romic +3 more
TL;DR: The types of learning problems commonly employed in medical image processing are introduced and an overview of commonly used deep learning methods are presented, including convolutional neural networks (CNNs), recurrent neural Networks (RNNs), and generative adversarial networks (GANs), with a focus on the image analysis task they are solving.
29
Deep Learning for Optical Sensor Applications: A Review
18 Jul 2023
TL;DR: In this article , the authors present recent studies integrating DL algorithms with optical sensor applications and highlight several directions for DL algorithms that promise a considerable impact on use for optical sensors applications, and provide new directions for the future development of related research.
16
References
Evolutionary artificial neural networks: a review
TL;DR: The advantages and disadvantages of using EAs to optimize ANNs are explained and the basic theories and algorithms for optimizing the weights, optimizing the network architecture and optimizing the learning rules are provided.
395
•Proceedings Article
ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning
Quoc V. Le,Alexandre Karpenko,Jiquan Ngiam,Andrew Y. Ng +3 more
- 12 Dec 2011
TL;DR: A robust soft reconstruction cost for ICA is proposed that allows us to learn highly overcomplete sparse features even on unwhitened data and reveals formal connections between ICA and sparse autoencoders, which have previously been observed only empirically.
A Sufficient Condition for Convergences of Adam and RMSProp
Fangyu Zou,Li Shen,Zequn Jie,Weizhong Zhang,Wei Liu +4 more
- 15 Jun 2019
TL;DR: In this paper, an alternative easy-to-check sufficient condition, which merely depends on the parameters of the base learning rate and combinations of historical second-order moments, was proposed to guarantee the global convergence of generic Adam/RMSProp for solving large-scale non-convex stochastic optimization.
•Proceedings Article
Unsupervised learning of distributions on binary vectors using two layer networks
Yoav Freund,David Haussler +1 more
- 02 Dec 1991
TL;DR: It is shown that arbitrary distributions of binary vectors can be approximated by the combination model and shown how the weight vectors in the model can be interpreted as high order correlation patterns among the input bits, and how the combination machine can be used as a mechanism for detecting these patterns.
Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video
Radu Tudor Ionescu,Fahad Shahbaz Khan,Mariana-Iuliana Georgescu,Ling Shao +3 more
- 15 Jun 2019
TL;DR: An unsupervised feature learning framework based on object-centric convolutional auto-encoders to encode both motion and appearance information is introduced and a supervised classification approach based on clustering the training samples into normality clusters is proposed.