Abstract: This letter proposes a background subtraction method for Bayer-pattern image sequences. The proposed method models the background in a Bayer-pattern domain using a mixture of Gaussians (MoG) and classifies the foreground in an interpolated red, green, and blue (RGB) domain. This method can achieve almost the same accuracy as MoG using RGB color images while maintaining computational resources (time and memory) similar to MoG using grayscale images. Experimental results show that the proposed method is a good solution to obtain high accuracy and low resource requirements simultaneously. This improvement is important for a low-level task like background subtraction since its accuracy affects the performance of high-level tasks, and is preferable for implementation in real-time embedded systems such as smart cameras.

Abstract: In digital image forensics, estimating the color filter array (CFA) pattern can be useful for digital camera identification. In this paper, we proposed the new method to estimate the CFA pattern of the digital cameras from a single image. Our method is based on the basic principal of CFA interpolation which fills an empty pixel using neighbor pixels. For each channel, we define the specific neighbor pattern and count the intermediate values. The CFA pattern is estimated by utilizing this counting information of three channels. The experimental results show that the proposed method achieves high accuracy with various camera models and CFA interpolation algorithms.

Abstract: This paper describes a new filter for impulse noise reduction in colour images which is aimed at improving the noise reduction capability of the classical vector median filter. The filter is inspired by the application of a vector marginal median filtering process over a selected group of pixels in each filtering window. This selection, which is based on the vector median, along with the application of the marginal median operation constitutes an adaptive process that leads to a more robust filter design. Also, the proposed method is able to process colour images without introducing colour artifacts. Experimental results show that the images filtered with the proposed method contain less noisy pixels than those obtained through the vector median filter.

Abstract: In this paper, we introduce a new demosaicing algorithm that can be used for various sensor images captured by digital cameras equipped with various red-green-blue color filter arrays. Our algorithm enhances the universal demosaicing algorithm of Lukac et al by defining a new spectral interpolation model that exploits not only the information on the color of pixels but also the relative distance between neighboring pixels within an image. Moreover, we include an edge-detection model that makes our algorithm adaptive and reduces the presence of color shifts and artifacts. A series of tests has been made on images of the Kodak database, and our algorithm performs better than the universal demosaicing algorithm with regard to both subjective and objective evaluation. The versatility of our demosaicing algorithm is also highlighted through an application to the issue of color image resampling, and we obtain conclusive experimental results.

Abstract: A pixel of an image sensor includes a color filter configured to pass visible wavelengths, and an infrared cut-off filter disposed on the color filter configured to cut off infrared wavelengths.

Abstract: In digital holographic interferometry, the resolution of the reconstructed hologram depends on the pixel size and pixel number of the sensor used for recording. When different wavelengths are simultaneously used as a luminous source for the interferometer, the shape and the overlapping of three filters of a color sensor strongly influence the three reconstructed images. This problem can be directly visualized in 2D Fourier planes on red, green and blue channels. To better understand this problem and to avoid parasitic images generated at the reconstruction, three different sensors have been tested: a CCD sensor equipped with a Bayer filter, a Foveon sensor and a 3CCD sensor. The first one is a Bayer mosaic where one half of the pixels detect the green color and only one-quarter detect the red or blue color. As the missing data are interpolated among color detection positions, offsets and artifacts are generated. The second one is a specific sensor constituted with three stacked photodiode layers. Its technology is different from that of the classical color mosaic sensor because each pixel location detects the three colors simultaneously. So, the three colors are recorded simultaneously with identical spatial resolution, which corresponds to the spatial resolution of the sensor. However, the spectral curve of the sensor is large along each wavelength since the color segmentation is based on the penetration depth of the photons in silicon. Finally, with a 3CCD sensor, each image is recorded on three different sensors with the same resolution. In order to test the sensor influence, we have developed a specific optical bench which allows the near wake flow around a circular cylinder at Mach 0.45 to be characterized. Finally, best results have been obtained with the 3CDD sensor.

Abstract: Microgrid imaging polarimeters consist of a focal plane array sensor with linear polarization filters of differing orientations overlaid at each pixel, similar in concept to the arrangement of spectral filters in a color CCD Bayer pattern camera However, unlike spectral color cameras, microgrid systems use polarimetrically modulated intensity measurements to reconstruct the Stokes vector at each point in an imaged scene Stokes reconstruction of imagery from these devices has traditionally been performed using linear filtering techniques While linear filtering strategies can yield reasonable estimates of the Stokes imagery, the filtering often results in loss of high frequency content in addition to introducing typical demosaicing artifacts (such as aliasing and zippering effects) Here we develop an adaptive demosaicing strategy based upon the concept of bilateral filtering as a means for reducing interpolation artifacts while preserving high frequency image content often removed by non-adaptive linear interpolators We demonstrate the demosaicing strategy and compare it against imagery estimated using other techniques on LWIR microgrid data1,2

Abstract: An image capture apparatus that includes an array of color filters for green, red, and magenta colors arranged over a semiconductor substrate in the manner of a primary color Bayer pattern except a magenta color replaces the blue color. Light passing through the magenta color filter is integrated separately in a magenta pixel for a shallow photodiode signal and a deep photodiode signal in a first photodiode and a deeper second photodiode in the substrate, respectively. A mezzanine photodiode may be disposed between the first and second photodiodes and held at a fixed voltage level or reset multiple times during charge integration. A red pixel value for the magenta pixel is a function of the deep photodiode signal and an adjacent red pixel's red pixel signal. A minimum exists in its derivative with respect to the former at a value of the former that varies with the latter.

Abstract: A complementary metal-oxide semiconductor (CMOS) image sensor and a pixel data readout method of the same are provided. The CMOS image sensor includes: a first readout line which outputs pixel data from a shared pixel group in an odd row of a column of a pixel array in a Bayer pattern during a horizontal period; and a second readout line which outputs pixel data from a shared pixel group in an even row of the column of the pixel array during the horizontal period, wherein pixel data output to the first and second readout lines during the horizontal period correspond to a basic Bayer pattern and pixels from which pixel data are read out in each column sequentially shifts in a column direction at each horizontal period.

Abstract: An image processing apparatus and a processing method thereof are provided. The image processing apparatus includes an image capturing module, an image separation module, an image stabilization module, a temporal noise reduction module, and a spatial noise reduction module. The image capturing module captures a plurality of Bayer pattern images. The image separation module decreases the Bayer pattern images in size and transforms them into a plurality of YCbCr format images. The image stabilization module receives Y channel images of the YCbCr format images and the Bayer pattern images to perform motion estimation, to produce a plurality of global motion vectors (GMVs). The temporal noise reduction module performs temporal blending process on the Bayer pattern images according to the GMVs, to produce first noise reduction images. The spatial noise reduction module performs 2-dimensional spatial noise reduction on the first noise reduction images to produce second noise reduction images.

Abstract: Image demosaicking (or demosaicing) is the interpolation problem of estimating complete color information for an image that has been captured through a color filter array (CFA), particularly on the Bayer pattern. While many complicated methods for demosaicking have been proposed, Malvar, He, and Cutler [5] showed that surprisingly good results are possible with a simple linear method using 5×5 filters.

Abstract: The utility of Compressed Sensing (CS) for demosaicing of digital images have been explored by few recent efforts. Most recently, a Compressive Demosaicing [3] framework, based on employing a random panchromatic Color Filter Array (CFA) at the sensing stage, has provided compelling CS-based demosaicing results by visually outperforming other leading techniques. Meanwhile, it is well known that the Bayer pattern is arguably the most popular CFA used in low-cost consumer digital cameras. In this paper, we explore and compare the Bayer and random panchromatic CFA structures using a generic approach for demosaicing of images based on recent advances in the field of CS. In particular, a key objective of this work is to provide a comparative analysis between these two CFA patterns (Bayer and random panchromatic) under the general umbrella of sparse recovery, which represents the cornerstone of CS-based decoding. We demonstrate the viability of the Bayer pattern under certain CS conditions. Meanwhile, we show that a random panchromatic CFA, which meets certain incoherence constraints, can visually outperform a Bayer based sparse recovery. As illustrated in our simulation results, a panchromatic CFA is more consistent in terms of providing better visual quality when tested on a wide range of color images.

Abstract: Multispectral method for air tract influence attenuation on basis of color detector with Bayer filter is introduced. The difference in position of target image in different colors allows compensating the air tract refraction.

Abstract: Most of the current digital cameras feature a single sensor design which limits the number of channels recorded at each pixel location to one. However, a color image is represented with three channels for each pixel. Color Filter Array (CFA) interpolation is the process of generating a full three channel color image from a single channel mosaicked input. We propose a simple edge strength filter to interpolate the missing color values adaptively. While the filter is readily applicable to the Bayer mosaic pattern, we argue that the same idea could be extended to other mosaic patterns and describe its application to the Lukac mosaic pattern. The proposed solution outperforms other available algorithms for the Lukac pattern in terms of both objective and subjective comparison.

Abstract: In high quality imaging even tiny distortions as small as a single pixel are visible and can not be accepted. Although the production quality of CMOS image sensors is very high, for reasonable yields we still need to accept some defect pixels and clusters of defects in large image sensors. In this paper we will compare compensation algorithms for raw image sensor data. We propose a new approach based on the sparsity assumption that outperforms existing defect compensation algorithms. Furthermore, our proposed interpolation algorithm is universal and not at all adapted to Bayer pattern images. It can directly be applied to any regular color filter pattern or gray scale image. Our examples show, that image sensors with large clusters of defects can still be used for the generation of high quality images.

Abstract: The present invention provides a method for efficiently removing noise of an image sensor. To this end, the present invention enables: an area to be divided into an edge area, a sensitivity difference generation area, and a pattern area relative to an input image of a Bayer pattern; and different noise removing methods to be applied according to each of the divided areas. By doing this, noise can be removed efficiently since the noise is adaptively removed in each area. Further, sensitivity differences between Gr and Gb are considered in such a manner that resolution can be improved.

Abstract: Demosaicking is the process of reconstructing a full color image from the incomplete color samples output of an image sensor overlaid with a color filter array. Most demosaicking algorithms usually suffer from visible color artifacts such as false colors. In this paper the hybrid use of frequency domain demosaicking with wavelet decomposition postprocessing is employed to reduce color artifacts usually found around edges. The algorithm is iterative and an adaptive stopping criterion is suggested. Results show an improvement in terms of visual and numerical comparisons using peak signal-to-noise ratio (PSNR) between the demosaicked image before and after postprocessing.

Abstract: Digital aerial cameras have replaced analog film cameras. An information contents corresponding to aerial photos up to 2010 was only possible with digital system cameras as Vexcel Ultracam and Z/I Imaging DMC or line scanning cameras as Leica Geosystems ADS80 or Jena Optronic JAS-150. Now with very large size CCDs the panchromatic band of the DMC II has between 140 and 256 Mega pixels from one CCD and this with excellent image geometry. With large size digital camera images more precise results as with analog photos can be reached. A strong development came for mid-format digital cameras having now up to 60 Megapixels. Some of them are also used in configurations of 2 up to 5 cameras, leading to a similar imaging capacity as the large format digital cameras, but not with the same geometric quality. Usually such cameras have to be supported at least by relative kinematic GPS-positioning or even inertial measurement units. Standard mid-format cameras have only one CCD-array, where 3 color bands can be generated by Bayer pattern, while system cameras usually have 4 color bands. Unmanned aerial vehicles (UAV) are becoming popular; partially they are equipped with tiny digital cameras with just 1.8µm pixel size, but still satisfying potential. With available 0.5m ground sampling distance (GSD) very high resolution optical satellites are competing with aerial cameras. Usually their image quality is on the same level. The absolute geo-reference in the range of 3m without use of ground control points (GCP) for GeoEye and Worldview images is satisfying for several applications. So for example a comparison of digital surface models (DSM) generated with IKONOS and with GeoEye stereo pairs was leading without GCP to discrepancies in X and Y below 10m and in the height below 1m. 0.5m up to 1.0m GSD allows the generation of 3D city models with satellite images. With semi global matching (SGM) sharp building contours can be generated. In near future even 0.3m GSD will be possible with GeoEye-2 and Cartosat-3. Just now there is still a limitation of the USA to distribute space images having below 0.5m GSD, but caused by the Indian competition this may change.

Abstract: Development of small pixels for high resolution image sensors implies a lot of challenges. A high level of performance should be guaranteed whereas the overall size must be reduced and so the degree of freedom in design and process. One key parameter of this constant improvement is the knowledge and the control of the crosstalk between pixels. In this paper, we present an advance in crosstalk characterization method based on the design of specific color patterns and the measurement of quantum efficiency. In a first part, we describe the color patterns designed to isolate one pixel or to simulate un-patterned colored pixels. These patterns have been implemented on test-chip and characterized. The second part deals with the characterization setup for quantum efficiency. Indeed, the use of spectral measurements allows us to discriminate pixels based on the color filter placed on top of them and to probe the crosstalk as a function of the depth in silicon, thanks to the photon absorption length variation with the wavelength. In the last part, results are presented showing the impact of color filters patterning, i.e. pixels in a Bayer pattern versus un-patterned pixels. The crosstalk directions and amplitudes are also analyzed in relation to pixel layout.

Abstract: This paper presents the novel demosaicing approach that is based upon region of interest analysis. The question of if only the reconstruction quality of areas where human gazes during identification process of security objects are crucial for quality of identification is solved. Two fixation density maps were constructed on the basis of the ground through visual attention data which was acquired during an eye tracking experiment. Two security scenes containing traffic signs were used as testing images. SSIM and VIF with subjective testing were used for evaluation of reconstruction quality. The results tackle the questions — if quality of identification mainly depends on the quality of the areas where human eye is attracted, areas of interest, and whether the quality of the remaining part of image perceived by peripheral vision is crucial for quality of identification. Furthermore, the results show that embedding the information relating to these visual fixation patterns into demosaicing procedure can preserve the perceived image quality and quality of identification, while the computational complexity of proposed approach can be low.

Abstract: A digital camera provided with a Bayer pattern single sensor needs color interpolation to reconstruct a full color image. To show high resolution image on a lower resolution display, it must then be down-sampled. These two steps influence each other, i.e., the color artifacts introduced in demosaicing may be magnified in subsequent down-sampling process and vice versa. Thanks to the fact that LCD displays are actually composed of separable subpixels, which can be individually addressed to achieve a higher effective apparent resolution. This paper presents an Adaptive Joint Demosaicing and Subpixel-based Down-sampling scheme (AJDSD) for single-sensor camera image, where the subpixel-based down-sampling is adaptively and directly applied in Bayer domain, without the process of demosaicing. Simulation results demonstrate that when compared with conventional “demosaicing-first and down-sampling-later” methods, AJDSD achieves superior performance improvement in terms of computational complexity. As for visual quality, AJDSD is more effective in preserving high frequency details, leading to much sharper and clearer results.

Abstract: A complementary metal-oxide semiconductor (CMOS) image sensor and a pixel data readout method of the same are provided. The CMOS image sensor includes: a first readout line which outputs pixel data from a shared pixel group in an odd row of a column of a pixel array in a Bayer pattern during a horizontal period; and a second readout line which outputs pixel data from a shared pixel group in an even row of the column of the pixel array during the horizontal period, wherein pixel data output to the first and second readout lines during the horizontal period correspond to a basic Bayer pattern and pixels from which pixel data are read out in each column sequentially shifts in a column direction at each horizontal period.

Abstract: Nowadays image sensor is an essential component in many multimedia devices, and it is covered by a color filter array to filter out specific color components at each pixel. We need a certain algorithm to combine those color components reconstructed a full color image from incomplete color samples output from an image sensor, which is called a demosaicking process. Most existing demosaicking algorithms are developed for ideal image sensors, but they do not work well for the practical cases because of dissimilar characteristics of each sensor. In this paper, we propose a new demosaicking algorithm in which the color filter characteristics are fully utilized to generate a good image. To demonstrate significance of our algorithm, we used a commerically available sensor, CBN385B, which is a sort of Honeycomb-style CFA(Color Filter Array) CCD image sensor. As a performance metric of the algorithm, PSNR(Peak Signal to Noise Ratio) and RGB distribution of the output image are used. We first implemented our algorithm in C-language for simulation on various input images. As a result, we could obtain much enhanced images whose PSNR was improved by 4~8 dB compared to the commonly idealized approaches, and we also could remove the inclined red property which was an unique characteristics of the image sensor(CBN385B).Then we implemented it in hardware to overcome its problem of computational complexity which made it operate slow in software. The hardware was verified on Spartan-3E FPGA(Field Programable Gate Array) to give almost the same performance as software, but in much faster execution time. The total logic gate count is 45K, and it handles 25 image frmaes per second.

Abstract: A widely used solution for cost effective and small cameras are single sensors with a color filter array (CFA). The most popular type of CFA is the Bayer color filter array. Only red, green and blue color components are passed to the sensor depending on their coordinates. In this paper a new method is presented to derive color images from noisy Bayer filtered data. The denoising and interpolation steps are done separately, using local techniques for color interpolation and non-local filters for denoising. The advantage of the method are less blurring and reduced color artifacts especially for highly textured images as compared to methods using local filtering for denoising.

Abstract: The present invention discloses a method of improving color quality in anaglyphs. Stereo pair images are viewed using a filter of one primary color on the side corresponding to the side of the same color primary filter used in recording the anaglyph. The other side, corresponding to the side recorded using color filter of the remaining primary colors, has only a fixed or adjustable neutral density filter that matches the luminosity to that of the one primary color filter. Retinal rivalry suppresses the one primary color image on the side of the less luminous neutral density filter image, thus generating full stereopis. Since the side with the neutral density filter perceives all of the primary colors, the color gamut is maintained.

Abstract: A method of interpolating a signal output from an image sensor including a pixel array having an M×N matrix as a basic pixel block where M and N are integers is provided. The method includes selecting a target pixel signal from among pixel signals output from the basic pixel block; and converting a pattern of a pixel signal output from the pixel array into a Bayer pattern by converting a pixel signal output from the basic pixel block into the Bayer pattern through an operation on the target pixel signal and a neighboring pixel signal of the target pixel signal and interpolating an output signal converted into the Bayer pattern.

Abstract: The image sensor includes a pixel array including a plurality of pixels arranged in a non-red-green-blue (RGB) Bayer pattern, an analog-to-digital converter configured to convert an analog pixel signal output from each of the pixels into a digital pixel signal, and an RGB converter configured to convert the digital pixel signal into an RGB Bayer signal. Accordingly, the image sensor is compatible with a universal image signal processor (ISP), which receives and processes RGB Bayer signals, without an additional compatible device or module.

Abstract: A method for image noise filtering is provided that includes receiving a Bayer domain image with four color channels, generating a hierarchical representation of the four color channels comprising a set of coefficient arrays at each level of the hierarchical representation, modifying the coefficient arrays of the color channels jointly to remove noise, and generating a noise filtered and edge enhanced Bayer domain image based on the jointly modified coefficient arrays.

Abstract: In quality assurance the inspection of geometric features of objects is one the most common tasks. There is a great variety of measuring principles. One of these is the measurement by electro-optical sensors with corresponding image processing. Image processing, on the other hand, is used for many different tasks as well. Examples are object recognition, colour measurement, scene interpretation and many more. Measurement of geometric features is one of many applications, special optimizations are therefore rarely applied for this particular applications needs. In image processing in general, the development over the last years was towards colour images or even more than three channels, so called “multi channel images”. One of the results of the advanced popularity of colour image processing is, that today some types of three channel cameras are not more expensive than their single channel counterparts. Even though these cameras are being used in system for measurement of geometric features, the algorithms used do not take advantage of the additional channels information. There are a lot of special colour image processing algorithms existing today, but there are very little concepts that address the application of measurement of geometries. In this thesis new approaches are being discussed to use the information delivered by colour image sensors in a way that the measurement of geometries in the image will be improved. Four different aspects of the chain of image processing will be addressed in this work. Two of them are applicable for all kinds of multi channel images and two are dedicated to special properties of the single most common colour image sensor type, the senor with attached colour filter array (CFA) with an arrangement according to B.E. Bayer. The two general multi channel approaches are: – Extraction of object edge information by means of a new image filter where the information of all available channels is used – High precision edge probing for those new filtered edge images with the aim of subpixel accurate edge position determination The two CFA-Sensor related aspects are: – A new “Demosaicing” algorithm to reconstruct the three channel image from the sensors raw data with special importance to geometrically correct edge reproduction – Choice for object illumination source where the interaction of the emission spectra of the source and spectral sensitivity of the senor is optimized to the needs of the designated application The new approaches presented in this thesis deliver a contribution to image processing for measurement of geometric features with multi channel images, i.e. colour images. With them, better results, respectively lower measurement uncertainty, can be achieved. While they are applicable in their presented state, they do not stand as completed system. They are meant as a new way, a concept, to utilise multi channel image data to enhance current measuring machines. In addition these concepts open up prospects to further improvement.