Optimizing an Image Coding Framework with Deep Learning-based Pre- and Post-Processing

TL;DR: Li et al. as mentioned in this paper proposed a new loss function which also considers a rate component, thus allowing to jointly minimize the rate and distortion, which can outperform the selected benchmarks, both classical and CNN-based.

Abstract: Convolutional neural networks (CNN) are a popular machine learning architecture used to address multiple image-based tasks from understanding to coding. This paper targets improving image compression efficiency by designing and optimizing an image coding framework where a standard image codec, e.g. JPEG, is combined with deep neural network based pre- and post-processing. While the pre-processing CNN targets simplifying the image to make it more amenable to compression, notably involving its down-sampling, the post-processing CNN targets enhancing the decoded image, also involving its up-sampling. To optimize the compression performance, the processing CNNs are trained involving a third CNN, so-called CNN-FakeCodec, which targets modeling the image codec output, since the encoder-decoder pair is not differentiable, thus not allowing any training. Since the available alternative coding solutions focus on minimizing the image distortion, this paper proposes a new loss function which also considers a rate component, thus allowing to jointly minimize the rate and distortion. The performance results show that the proposed coding solutions can outperform the selected benchmarks, both classical and CNN-based.