Abstract: This paper presents a novel multidirectional weighted interpolation algorithm for color filter array interpolation. Our proposed method has two contributions to demosaicking. First, different from conventional interpolation methods based on two directions or four directions, the proposed method exploits to greater degree correlations among neighboring pixels along eight directions to improve the interpolation performance. Second, we propose an efficient postprocessing method to reduce interpolation artifacts based on the color difference planes. Compared with conventional state-of-the-art demosaicking algorithms, our experimental results show the proposed algorithm provides superior performance in both objective and subjective image quality. Furthermore, this implementation has moderate computational complexity.

Abstract: Single-sensor color cameras, which classically use a color filter array (CFA) to sample RGB channels, have recently been extended to the multispectral domain. To sample more than three wavelength bands, such systems use a multispectral filter array (MSFA) that provides a raw image in which a single channel level is available at each pixel. A demosaicing procedure is then needed to estimate a multispectral image with full spectral resolution. In this paper, we propose a new demosaicing method that takes spectral and spatial correlations into account by estimating the level for each channel. Experimental results show that it provides estimated images of better quality than classical methods.

Abstract: Digital image sensor outputs usually must be transformed to suit the human visual system. This color correction amplifies noise, thus reducing the signal-to-noise ratio (SNR) of the image. In subdiffraction-limit (SDL) pixels, where optical and carrier cross talk can be substantial, this problem can become significant when conventional color filter arrays (CFAs) such as the Bayer patterns (RGB and CMY) are used. We present the design and analysis of new color filter array patterns for improving the color error and SNR deterioration caused by cross talk in these SDL pixels. We demonstrate an improvement in the color reproduction accuracy and SNR in high cross-talk conditions. Finally, we investigate the trade-off between color accuracy and SNR for the different CFA patterns.

Abstract: In this letter, we proposed a new framework for color image demosaicking by using different strategies on green (G) and red/blue (R/B) components. Firstly, for G component, the missing samples are estimated by eight-direction weighted interpolation via exploiting spatial and spectral correlations of neighboring pixels. The G plane can be well reconstructed by considering the joint contribution of pre-estimations along eight interpolation directions with different weighting factors. Secondly, we estimate R/B components using guided filter with the reconstructed G plane as guidance image. Simulation results verify that, the proposed framework performs better than state-of-the-art demosaicking methods in term of color peak signal-to-noise ratio (CPSNR) and feature similarity index measure (FSIM), as well as higher visual quality.

Abstract: An example image capture device includes an image sensor having diodes for sensing light from a target scene, a color filter array disposed above the diodes and including color filters each positioned over one of the diodes, single-diode microlenses positioned above some color filters arranged in a Bayer pattern, and multi-diode microlenses each positioned above at least two adjacent color filters that pass the same wavelengths of light to corresponding adjacent diodes below the color filters, each multi-diode microlens formed such that light incident in a first direction is collected one of the adjacent diodes and light incident in a second direction is collected in another of adjacent diodes. An image signal processor of the image capture device can perform phase detection autofocus using signals received from the adjacent diodes and can interpolate color values for the adjacent diodes.

Abstract: Most common cameras use a CCD sensor device measuring a single color per pixel. Demosaicking is the interpolation process by which one can infer a full color image from such a matrix of values, thus interpolating the two missing components per pixel. Most demosaicking methods take advantage of inter-channel correlation locally selecting the best interpolation direction. The obtained results look convincing except when local geometry cannot be inferred from neighboring pixels or channel correlation is low. In these cases, these algorithms create interpolation artifacts such as zipper effect or color aliasing. This paper discusses the implementation details of the algorithm proposed in [J. Duran, A. Buades, “Self-Similarity and Spectral Correlation Adaptive Algorithm for Color Demosaicking”, IEEE Transactions on Image Processing, 23(9), pp. 4031–4040, 2014]. The proposed method involves nonlocal image self-similarity in order to reduce interpolation artifacts when local geometry is ambiguous. It further introduces a clear and intuitive manner of balancing how much channel-correlation must be taken advantage of. Source Code An ANSI C source code implementation of the described algorithms is accessible at the IPOL web page of this article 1 , together with an on-line demo.

Abstract: Digital cameras capture images through different Color Filter Arrays and then reconstruct the full color image. Each CFA pixel only captures one primary color component; the other primary components will be estimated using information from neighboring pixels. During the demosaicking algorithm, the two unknown color components will be estimated at each pixel location. Most of the demosaicking algorithms use the RGB Bayer CFA pattern with Red, Green and Blue filters. The least-Squares Luma-Chroma demultiplexing method is a state of the art demosaicking method for the Bayer CFA. In this paper we develop a new demosaicking algorithm using the Kodak-RGBW CFA. This particular CFA reduces noise and improves the quality of the reconstructed images by adding white pixels. We have applied non-adaptive and adaptive demosaicking method using the Kodak-RGBW CFA on the standard Kodak image dataset and the results have been compared with previous work.

Abstract: Digital still cameras use image sensors to capture images. To reduce the size of digital still cameras, a single image sensor instead of three image sensors should be used. A single image sensor overlaid with a color filter array (CFA) that filters color channels only captures one color channel at each pixel; thus, the other missing color channels at each pixel must be reproduced from neighboring pixels. This procedure is known as CFA interpolation or demosaicking. In this paper, a novel CFA interpolation method is proposed to reduce false color artifacts and improve the quality of restored images. First, an edge detection method is used to interpolate the initial missing color channels. Second, the high correlations between wavelet subbands of the different color channels are explored to obtain accurate color values of the estimated green channel. Finally, the high-frequency subbands of red and blue channels are iteratively updated to reduce false color artifacts. Experimental results show that our proposed method preserves edges and restores images with high quality.

Abstract: Demosaicing is the process of interpolating the output from a single chip colour filter array sensor to form a full colour image. In hardware, the simplest algorithms: zero order hold and bilinear interpolation, are commonly used because of their simplicity and low resource requirements. State of the art algorithms are difficult to implement in hardware because of their complex access patterns. This paper explores the streamed implementation of a higher order interpolation filter, with a weighted median classifier. Although this comes at a cost of a factor of 10 increase in hardware resources, and a reduction in maximum pixel clock frequency by 30%, this state of the art algorithm gives considerably improved images of 11.2 dB in peak signal to noise ratio with a considerable reduction in interpolation artifacts. For real-time applications where image quality is critical, an implementation of such an advanced demosaicing algorithm on FPGA is essential.

Abstract: The advent of efficient short range radio communication coupled with advancement in miniaturization of computing devices has enabled the development of wireless multimedia sensor network (WMSN). Energy is one of the scarcest resources in such networks, especially it is scarce in transmitting multimedia data. This work presents an energy-saving image compression scheme. The nodes in WMSN are equipped with Colour Filter Array (CFA) using Bayer Pattern. Considering alternate column dropping of pixel array of the captured image is one of the viable techniques for low-overhead image comprssion, Bayer CFA lacks in features by means of which image pixel may be reconstructed using its neighbouring pixel information. So primarily we design a new CFA (NCFA) to overcome the said problems of Bayer CFA. Subsequently, the compression scheme is proposed where the alternate columns of the pixel array of each of the macroblocks of an image captured through NCFA, are dropped. At the high-end receiving node (e.g. sink), image is reconstructed from the sequentially transmitted macroblocks. Performance of the scheme is evaluated and compared through qualitative and quantitative analyses. The results show the present scheme's dominance over all the variants of the competing scheme in terms of both energy-efficiency and reconstruction quality.

Abstract: The invention provides an image sensor structure for realizing physical consolidation of pixels The image sensor structure comprises a pixel unit circuit array and filters, the filters cover all pixels in the array, and the array is formed by combining and extending multiple red, blue and green pixel unit circuits which meet the minimum BAYER pattern Each pixel unit circuit comprises four pixels, one pixel unit circuit is only covered by the filter in one color, and the pixel unit circuits are arranged at intervals according to the minimum square array The invention also provides a method for realizing physical consolidation of the pixels By controlling peripheral circuits in the pixel unit circuits, signals of every four pixels are read at the same time, and physical consolidation is completed The structure and method realize direct consolidation of photoelectrons through physical consolidation, errors caused by introduction of digital-to-analogue conversion are avoided, special data algorithms and complex peripheral circuits are not needed, and the sizes of the pixel units can be further reduced conveniently through a new framework with transistor pixels shared

Abstract: Imaging systems based on multispectral filter arrays(MSFA) can simultaneously acquire wide spectral information. A MSFA image sensor with R, G, B, and near-infrared(NIR) filters can obtain the mixed spectral information of visible bands and that of the NIR bands. Since the color filter materials used in MSFA sensors were almost transparent in the NIR range, the observed colors of multispectral images were degraded by the additional NIR spectral band information. To overcome this color degradation, a new signal processing approach is needed to separate the spectral information of visible bands from the mixed spectral information. In this paper, a color restoration method for imaging systems based on MSFA sensors is proposed. The proposed method restores the received image by removing NIR band spectral information from the mixed wide spectral information. To remove additional spectral information of the NIR band, spectral estimation and spectral decomposition were performed based on the spectral characteristics of the MSFA sensor. The experimental results show that the proposed method restored color information by removing unwanted NIR contributions to the RGB color channels.

Abstract: In this paper, an image fusion algorithm is proposed for a multi-aperture camera. Such camera is a worthy alternative to traditional Bayer filter camera in terms of image quality, camera size and camera features. The camera consists of several camera units, each having dedicated optics and color filter. The main challenge of a multi-aperture camera arises from the fact that each camera unit has a slightly different viewpoint. Our image fusion algorithm corrects the parallax error between the sub-images using a disparity map, which is estimated from the multi-spectral images. We improve the disparity estimation by combining matching costs over multiple views with help of trifocal tensors. Images are matched using two alternative matching costs, mutual information and Census transform. We also compare two different disparity estimation methods, graph cuts and semi-global matching. The results show that the overall quality of the fused images is near the reference images.

Abstract: The invention provides an image processing method. The image processing method includes the following steps that firstly, whether a first image generated by a color light filter array based on a bayer pattern is provided with a moire region is detected; secondly, color gamut conversion is conducted on the first image to generate a second image based on a red, green and blue region; thirdly, if the first image is provided with the moire region, compensation is conducted on multiple sub pixels for the moire portion of the second image to generate a third image, wherein the moire region of the second image corresponds to the moire region of the first image.

Abstract: Color Filter Arrays (CFA) used by single sensor cameras captures single color information at each pixel location. The process of estimating the missing color samples to reconstruct a full color image is called color filter array interpolation or demosaicing. Demosaicing the CFA images without denoising leads to demosaicing artifacts that will reduce the perceptual quality of the image. This paper presents a denoising before demosaicing strategy for denoising of CFA images captured by single sensor cameras. The images captured by the camera using the Bayer pattern are first denoised by Principal Component Analysis and followed by multiscale gradient based demosaicing to preserve the edges and details. The demosaicing strategy is followed by a false color suppression method to remove the residual demosaicing artifacts.

Abstract: A method for demosaicing, performed by a processing unit, at least containing: acquiring a frame with a Bayer pattern, wherein the Bayer pattern has alternating red (R), green (G) and blue (B) pixels; calculating a green (RG) value for each R pixel; calculating a green (BG) value for each B pixel; calculating a blue (RB) value for each R pixel; and calculating a red (BR) value for each B pixel. The step of calculating an RG or BG value for each R or B pixel at least contains: selecting one of a first interpolation algorithm and a second interpolation algorithm according to chrominance differences between the R or B pixel and surrounding pixels in a horizontal direction and a vertical direction; and using the selected interpolation algorithm to calculate the RG or BG value for the R or B pixel.

Abstract: Some embodiments provide an image sensor color filter array pattern that mitigates and/or minimizes the impact of optical and carrier crosstalk on color reproduction accuracy and/or signal-to-noise, the color filter array comprising a plurality of kernels, wherein each kernel has an identical configuration of color filter elements comprising primary color filter elements corresponding to at least three respective different primary colors, and a plurality of secondary color filter elements. A respective one of the secondary color filter elements is disposed as a nearest neighbor to and between every pair of primary color filter elements of different colors in the kernel, with the respective secondary color filter element representing a secondary color that is a combination of the different colors of the primary color filter elements that are nearest neighbors to the respective secondary color filter element.

Abstract: The recent technology in digital cameras uses single CCD or CMOS sensors as a substitute to three separate RGB sensors Color camera consists of single CCD or CMOS sensor with an overlaid Color Filter Array (CFA) This paper proposes a novel approach of CFA demosaicing In proposed approach, bilinear interpolation with sharpening filter is used for CFA demosaicing The proposed work tested on Virtex 5 FPGA in real time mode as well as on true color test image database Xilinx System Generator (XSG) has been used for system modeling and design implemented on target device FPGA The results of proposed FPGA based work improved than software based simple bilinear interpolation, edge directed interpolation and adaptive color plane interpolation methods and comparative to modified adaptive color plane interpolation method The proposed approach has low computational complexity and high image processing speed; hence the design can be implemented in real time low cost embedded system applications

Abstract: One method to acquire multispectral images is to sequentially capture a series of images where each image contains information from a different bandwidth of light. Another method is to use a series of beamsplitters and dichroic filters to guide different bandwidths of light onto different cameras. However, these methods are very time consuming and expensive and perform poorly in dynamic scenes or when observing transient phenomena. An alternative strategy to capturing multispectral data is to infer this data using sparse spectral reflectance measurements captured using an imaging device with overlapping bandpass filters, such as a consumer digital camera using a Bayer filter pattern. Currently the only method of inferring dense reflectance spectra is the Wiener adaptive filter, which makes Gaussian assumptions about the data. However, these assumptions may not always hold true for all data. We propose a new technique to infer dense reflectance spectra from sparse spectral measurements through the use of a non-linear regression model. The non-linear regression model used in this technique is the random forest model, which is an ensemble of decision trees and trained via the spectral characterization of the optical imaging system and spectral data pair generation. This model is then evaluated by spectrally characterizing different patches on the Macbeth color chart, as well as by reconstructing inferred multispectral images. Results show that the proposed technique can produce inferred dense reflectance spectra that correlate well with the true dense reflectance spectra, which illustrates the merits of the technique.

Abstract: The role of an optical low-pass filter (OLPF) in a digital still camera is to remove the high spatial frequencies that cause aliasing, thereby enhancing the image quality. However, this also causes some loss of detail. Yet, when an image is captured without the OLPF, moiré generally appears in the high spatial frequency region of the image. Accordingly, this paper presents a moiré reduction method that allows omission of the OLPF. Since most digital still cameras use a CCD or a CMOS with a Bayer pattern, moiré patterns and color artifacts are simultaneously induced by aliasing at high spatial frequencies. Therefore, in this study, moiré reduction is performed in both the luminance channel to remove the moiré patterns and the color channel to reduce color smearing. To detect the moiré patterns, the spatial frequency response (SFR) of the camera is first analyzed. The moiré regions are identified using patterns related to the SFR of the camera and then analyzed in the frequency domain. The moiré patterns are reduced by removing their frequency components, represented by the inflection point between the high-frequency and DC components in the moiré region. To reduce the color smearing, color changing regions are detected using the color variation ratios for the RGB channels and then corrected by multiplying with the average surrounding colors. Experiments confirm that the proposed method is able to reduce the moiré in both the luminance and color channels, while also preserving the detail. key words: optical low pass filter, moiré reduction, color smearing, image quality

Abstract: Color filter array (CFA) used in CCD and CMOS-sensor to acquire color images determinate the end quality of the image. In this paper the question of the requirements for the CFA to provide both high sensitivity and resolution is discussed. By optimizing the key parameters of the CFA new filters are obtained. The computer simulation proved that proposed CFA outperform known CFA by meanings of objective and subjective image quality.

Abstract: This paper presents an efficient hardware architecture to perform the real-time correction of the barrel distortion in wide-angle cameras. Applied to the single-sensor cameras employing the Bayer pattern as the color filter array (CFA), the proposed architecture performs the backward mapping process once for each pixel location in the Bayer pattern and the sub-pixel image resampling process for each color channel considering the CFA. As a result, the proposed architecture performs the barrel distortion correction jointly with the color demosaicking effectively. A prototype of the BDC processor based on the proposed architecture is implemented with 53.3K logic gates in a 0.18μm CMOS technology and its correction speed is 311M pixels/s, which shows that the proposed architecture has low complexity even with such versatile functionality.

Abstract: An image sensor, and an apparatus and method of acquiring an image by using the image sensor are provided. The image sensor includes a color filter having an array of a plurality of types of color filter elements, where each of the color filter elements transmits visible light in a certain wavelength band and blocks visible light outside the certain wavelength band; a photoelectric conversion cell array that detects light that has been transmitted through the color filter; and a modulator, disposed on the photoelectric conversion cell array, which changes a rate of light transmitted to the photoelectric conversion cell array based on an applied voltage.

Abstract: The coded aperture snapshot spectral imager (CASSI) system has been demonstrated to produce spectral images using just a single snapshot taken by a monochrome image sensor. In this paper, we introduce the use of color image sensors in CASSI to include the spectral sensitivity of the image sensor pixels to account for color and then investigate the impact on image quality. We analyze the use of the traditional Bayer color filter array image sensor, and the novel Foveon image sensor which stacks red, green, and blue pixels on top of one another. Several simulations on different 3D spatio-spectral databases show improvements of up to 3 dBs in terms of PSNR over traditional monochrome sensors.

Abstract: An image sensor includes an inorganic color filter, an organic color filter, and a metal pattern. The inorganic color filter is on a support substrate. The organic color filter is on the support substrate. The organic color filter is spaced apart from the inorganic color filter. The metal pattern is between the inorganic color filter and the organic color filter.