Open AccessPosted Content
Hierarchical Predictive Coding Models in a Deep-Learning Framework.
Matin Hosseini,Anthony S. Maida +1 more
TL;DR: This paper reviews some of the more well known models of Bayesian predictive coding and analyzes module connectivity and patterns of information transfer, seeking to find general principles used across the models.
read more
Abstract: Bayesian predictive coding is a putative neuromorphic method for acquiring higher-level neural representations to account for sensory input. Although originating in the neuroscience community, there are also efforts in the machine learning community to study these models. This paper reviews some of the more well known models. Our review analyzes module connectivity and patterns of information transfer, seeking to find general principles used across the models. We also survey some recent attempts to cast these models within a deep learning framework. A defining feature of Bayesian predictive coding is that it uses top-down, reconstructive mechanisms to predict incoming sensory inputs or their lower-level representations. Discrepancies between the predicted and the actual inputs, known as prediction errors, then give rise to future learning that refines and improves the predictive accuracy of learned higher-level representations. Predictive coding models intended to describe computations in the neocortex emerged prior to the development of deep learning and used a communication structure between modules that we name the Rao-Ballard protocol. This protocol was derived from a Bayesian generative model with some rather strong statistical assumptions. The RB protocol provides a rubric to assess the fidelity of deep learning models that claim to implement predictive coding.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Predictive Coding: a Theoretical and Experimental Review
TL;DR: Predictive coding offers a potentially unifying account of cortical function as mentioned in this paper, postulating that the core function of the brain is to minimize prediction errors with respect to a generative model of the world.
7
Inception Recurrent Neural Network Architecture for Video Frame Prediction
TL;DR: A novel Inception-based convolutional recurrent neural network (RNN) as an enhancement to a basic gated Convolutional RNN is proposed, which replaces the single-size kernel in the convolutionAL RNN with Inceptions-like multi-channel kernels.
4
PredNet⊕GAN: A Higher-Level Induction of Predictive Coding into Adversarial Setting Leading to a Semi/Pseudo-GAN Perspective at a Lower Level
01 Apr 2023
TL;DR: In this paper , the predictive coding neural network model (PredNet) was applied in an adversarial setting of Wasserstein and the Conditional-Wasserstein nature, and the results from their experiments seem to substantiate a new perspective on predictive coding theory.
PredNet⊕GAN: A Higher-Level Induction of Predictive Coding into Adversarial Setting Leading to a Semi/Pseudo-GAN Perspective at a Lower Level
SaiDatta Mikkilineni,Taylor D. Privat,Michael W. Totaro +2 more
- 01 Apr 2023
TL;DR: In this article , the predictive coding neural network model (PredNet) was applied in an adversarial setting of Wasserstein and the Conditional-Wasserstein nature, and the results from their experiments seem to substantiate a new perspective on predictive coding theory.
Inconsistent illusory motion in predictive coding deep neural networks
TL;DR: In this article , a pretrained PredNet predicted illusory motion for all subcomponents of the Rotating Snakes pattern, consistent with human observers, but no network predicted motion for greyscale variants of the rotating snakes pattern.
References
•Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
- 04 Sep 2014
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
102.6K
ImageNet classification with deep convolutional neural networks
TL;DR: A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Image quality assessment: from error visibility to structural similarity
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
•Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
- 01 Jan 2015
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
51.9K