Spectral Image Visualization Using Generative Adversarial Networks

TL;DR: Wang et al. as discussed by the authors presented a novel visualization method based on generative adversarial network (GAN) to display spectral images in natural colors, in which a structure loss and an adversarial loss are combined to form a new loss function.

Abstract: Spectral images captured by satellites and radio-telescopes are analyzed to obtain information about geological compositions distributions, distant asters as well as undersea terrain. Spectral images usually contain tens to hundreds of continuous narrow spectral bands, so the visualization based on information fusion and dimensional reduction is required for proper display on a trichromatic monitor which is important for spectral image processing and analysis system. The visualizations of spectral images should preserve as much information as possible from the original signal and facilitate image interpretation. However, most of the existing visualization methods display spectral images in false colors, which contradicts with human’s expectation and experience. In this paper, we present a novel visualization method based on generative adversarial network (GAN) to display spectral images in natural colors, in which a structure loss and an adversarial loss are combined to form a new loss function. The adversarial loss fits the visualized image to the natural image distribution using a discriminator network that is trained to distinguish false-color images from natural-color images. At the same time, we use an improved cycle loss as the structure constraint to guarantee structure consistency. Experimental results show that our method is able to generate structure-preserved and natural-looking visualizations.