Efficient-PrototypicalNet with self knowledge distillation for few-shot learning

TL;DR: Wang et al. as mentioned in this paper proposed a Motion Stimulation (MS) block, which is specifically designed to mine dynamic clues of the local regions autonomously from adjacent frames, which can be directly and conveniently integrated into existing video backbones to enhance the ability of compositional generalization for action recognition algorithms.

TL;DR: This work proposes SSL-ProtoNet, a self-supervised learning approach that leverages prototypical networks and knowledge distillation to enhance sample discrimination and few-shot learning performance, outperforming state-of-the-art methods on miniImageNet, tieredImageNet, and CIFAR-FS datasets.

TL;DR: A novel data-free knowledge distillation method (Spaceship-Net) based on channel-wise feature exchange (CFE) and multi-scale spatial activation region consistency (mSARC) constraint to assure the student network can imitate not only the logit output but also the spatialactivation region of the teacher network in order to alleviate the influence of unwanted noises in diverse synthetic images on distillation learning.

TL;DR: Self-supervised contrastive learning (SCL) as discussed by the authors enriched the model representation with multiple self-supervision objectives to improve the performance of few-shot image classification with a limited number of base class samples.

TL;DR: This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.

TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.

TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.

TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.