Multispectral segmentation

Abstract: Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15, 000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de.

Abstract: The authors present an unsupervised clustering technique for multispectral segmentation of magnetic resonance (MR) images of the human brain. Their scheme utilizes the Self Organizing Feature Map (SOFM) artificial neural network for feature mapping and generates a set of codebook vectors. By extending the network with an additional layer the map will be classified and each tissue class will be labeled. An algorithm has been developed for extracting the cerebrum from the head scan prior to the segmentation. Extracting the cerebrum is performed by stripping away the skull pixels from the T2 image. Three tissue types of the brain: white matter, gray matter and cerebral spinal fluid (CSF) are segmented accurately. To compare the results with other conventional approaches the authors applied the c-means algorithm to the problem.

Abstract: Classification algorithms based on single-pixel analysis often do not give the desired result when applied to high-spatial-resolution remote-sensing data. In such cases, classification algorithms based on object-oriented image segmentation are needed. There are many segmentation algorithms in the literature, but few have been applied in urban studies to classify a high-spatial-resolution remote-sensing image. Furthermore, the user must specify the spectral and spatial parameters that are data dependent. In this paper, we propose an automatic multispectral segmentation algorithm inspired by the specific idea of guiding a classification process for a high-spatial-resolution remote-sensing image of an urban area using an existing digital map of the same area. The classification results could be used, for example, for high-scale database updating or change-detection studies. The algorithm developed uses digital maps and spectral data as inputs. It generates the segmentation parameters automatically. The algorithm is able to provide a segmented image with accuracy greater than 90%. The segmentation results are then used in a rule-based classification using spectral, geometric, textural, and contextual information. The classification accuracy of the proposed rule-based classification is at least 17% greater than the maximum-likelihood classification results. Results and future improvements will be discussed.