TL;DR: A gradient-based registration algorithm is utilized to estimate the shifts between the acquired frames and then a weighted nearest-neighbor approach is used for placing the frames onto a uniform grid to form a final high-resolution image.
Abstract: Forward looking infrared (FLIR) detector arrays generally produce spatially undersampled images because the FLIR arrays cannot be made dense enough to yield a sufficiently high spatial sampling frequency. Multi-frame techniques, such as microscanning, are an effective means of reducing aliasing and increasing resolution in images produced by staring imaging systems. These techniques involve interlacing a set of image frames that have been shifted with respect to each other during acquisition. The FLIR system is mounted on a moving platform, such as an aircraft, and the vibrations associated with the platform are used to generate the shifts. Since a fixed number of image frames is required, and the shifts are random, the acquired frames will not fall on a uniformly spaced grid. Furthermore, some of the acquired frames may have almost similar shifts thus making them unusable for high-resolution image reconstruction. In this paper, we utilize a gradient-based registration algorithm to estimate the shifts between the acquired frames and then use a weighted nearest-neighbor approach for placing the frames onto a uniform grid to form a final high-resolution image. Blurring by the detector and optics of the imaging system limits the increase in image resolution when microscanning is attempted at sub-pixel movements of less than half the detector width. We resolve this difficulty by the application of the Wiener filter, designed using the modulation transfer function (MTF) of the imaging system, to the high-resolution image. Simulation and experimental results are presented to verify the effectiveness of the proposed technique. The techniques proposed herein are significantly faster than alternate techniques, and are found to be especially suitable for real-time applications.
TL;DR: In this paper, a writable grating was used to steer broad-spectral-band radiation for use in passive sensors in microscan systems, where the dispersion that is present is less than the resolution of the sensor considered.
Abstract: Liquid crystal writable grating technology is being developed for beam steering in laser radar systems. We consider the ability of writable gratings to steer broad-spectral-band radiation for use in passive sensors. We find that there is potential for these devices in microscan systems because there is little or no dispersion for the small scan angles required in microscanning. The dispersion that is present is less than the resolution of the sensor considered here. For large angle steering we find that dispersion correction or a narrowing of the spectral bandwidth is required. The degradation in sensitivity resulting from narrowing the spectral bandwidth is considered. We find that a high-quantum-efficiency step-stare sensor with a two-dimensional focal plane array responsive over a narrow spectral width can achieve the same sensitivity as current linear scanning sensors while being able to steer the field of view (FOV) over a larger field of regard with no moving parts. Approaches for dispersion correction and postdetection correction are discussed. A promising approach for steering a narrow FOV with broad spectral content and good resolution is described.
TL;DR: In this paper, the authors introduce and analyze techniques for the reduction of aliasing signal energy in a staring infrared imaging system, referred to as microscanning, exploit subpixel shifts between time frames of an image sequence.
Abstract: We introduce and analyze techniques for the reduction of aliased signal energy in a staring infrared imaging system. A standard staring system uses a fixed two-dimensional detector array that corresponds to a fixed spatial sampling frequency determined by the detector pitch or spacing. Aliasing will occur when sampling a scene containing spatial frequencies exceeding half the sampling frequency. This aliasing can significantly degrade the image quality. The aliasing reduction schemes presented here, referred to as microscanning, exploit subpixel shifts between time frames of an image sequence. These multiple images are used to reconstruct a single frame with reduced aliasing. If the shifts are controlled, using a mirror or beam steerer for example, one can obtain a uniformly sampled microscanned image. The reconstruction in this case can be accomplished by a straightforward interlacing of the time frames. If the shifts are uncontrolled, the effective sampling may be nonuniform and reconstruction becomes more complex. A sampling
model is developed and the aliased signal energy is analyzed for the microscanning techniques. Finally, a number of experimental results are presented that illustrate the perlormance of the microscanning methods.
TL;DR: In this article, an apparatus and method to determine the surface orientation of objects in a field of view is provided by utilizing an array of polarizers and a means for microscanning an image of the objects over the polarizer array.
Abstract: An apparatus and method to determine the surface orientation of objects in a field of view is provided by utilizing an array of polarizers and a means for microscanning an image of the objects over the polarizer array. In the preferred embodiment, a sequence of three image frames is captured using a focal plane array of photodetectors. Between frames the image is displaced by a distance equal to a polarizer array element. By combining the signals recorded in the three image frames, the intensity, percent of linear polarization, and angle of the polarization plane can be determined for radiation from each point on the object. The intensity can be used to determine the temperature at a corresponding point on the object. The percent of linear polarization and angle of the polarization plane can be used to determine the surface orientation at a corresponding point on the object. Surface orientation data from different points on the object can be combined to determine the object's shape and pose. Images of the Stokes parameters can be captured and viewed at video frequency. In an alternative embodiment, multi-spectral images can be captured for objects with point source resolution. Potential applications are in robotic vision, machine vision, computer vision, remote sensing, and infrared missile seekers. Other applications are detection and recognition of objects, automatic object recognition, and surveillance. This method of sensing is potentially useful in autonomous navigation and obstacle avoidance systems in automobiles and automated manufacturing and quality control systems.
TL;DR: It is demonstrated that the registration is a viable real-time algorithm that is suitable for applications involving small image shifts (i.e. less than one detector element) and the reconstruction program gives dramatic improvements in the image's resolution and does well in handling the aliased information.
Abstract: Staring infrared detectors often produce low resolution images because the technology does not exist to produce higher resolution arrays with sufficient spatial sampling intervals. A proven approach to combat this difficulty involves recording multiple frames that have been optically shifted onto a high-resolution grid pattern and then combined together into a single high resolution image. This process is known as microscanning. In fact, if the infrared (IR) imaging system is mounted on a moving platform, the normal vibrations associated with the platform's movement can be exploited to generate shifts in the acquired images. We present an algorithm that can register this temporal image sequence at the sub-pixel level and then reconstruct a high resolution image from the shifted frames. The proposed algorithm uses a gradient based shift estimator which provides shift information for each of the recorded frames. The reconstruction algorithm is based on a technique of high resolution image reconstruction by solving a series of linear equations in the frequency domain. In this paper, we review the theory behind the registration and reconstruction algorithms and their limitations. We demonstrate that the registration is a viable real-time algorithm that is suitable for applications involving small image shifts (i.e. less than one detector element). We also show that the reconstruction program gives dramatic improvements in the image's resolution and does well in handling the aliased information.