Models for Patch Based Image Restoration

TL;DR: A multi layer graphical model is proposed which unifies the low level vision task of restoration, and the high levelvision task of recognition in a cooperative framework, and is an interconnected two layer Markov Random Field.

Abstract: In this paper we present a supervised learning approach for object-category specific restoration, recognition and segmentation of images which are blurred using an unknown kernel. The feature of this work is a multi layer graphical model which unifies the low level vision task of restoration, and the high level vision task of recognition in a cooperative framework. Proposed graphical model is an interconnected two layer Markov Random Field. The restoration layer accounts for the compatibility between sharp and blurred patches, and models the association between adjacent patches in the sharp image. The recognition layer encodes the patch location and class. The potentials are represented using non-parametric kernel densities and are learnt from the training data. Inference is performed using nonparametric belief propagation. We propose a similar model for super-resolution from multiple frames, and suggest the use of ordinal regression for sub-pixel shift estimation to address the registration issues. Experiments demonstrate the effectiveness of proposed models for the restoration and recognition of blurred license plate and face images.