Abstract: In this letter, we present a new motion estimation algorithm for frame rate up-conversion. The proposed dual motion estimation algorithm enhances the estimation accuracy of motion vectors by using the unidirectional and bidirectional matching ratios of blocks in the previous and current frames. In addition, the proposed motion estimation approach uses motion vector validity to evaluate the accuracy of motion vectors thereby avoiding false motion vectors. In experiments using benchmark image sequences, the proposed motion estimation algorithm improved the average peak signal-to-noise ratio of interpolated frames by up to 2.272 dB, when compared to conventional motion estimation algorithms. For the comparison of the perceptual image quality using the structural similarity, the average value of the proposed dual motion estimation was by up to 0.062 higher than those of the conventional algorithms.

Abstract: The invention provides a system method for object detection and tracking in a video stream. Frames of the video stream are divided into regions of interest and a probability is calculated for each region of interest that the region contains at least a portion of an object to be tracked. The regions of interest in each frame are then classified based on the calculated probabilities. A region of interest (RI) frame is then constructed for each video frame that reports the classification of regions of interest in the video frame. Two or more RI frames are then compared in order to determine a motion of the object. The invention also provides a system executing the method of the invention, as well as a device comprising the system. The device may be for example, a portable computer, a mobile telephone, or an entertainment device.

Abstract: A video encoding system encodes video streams for multiple bit rate video streaming using an approach that permits the encoded resolution to vary based, at least in part, on motion complexity. The video encoding system dynamically decides an encoding resolution for segments of the multiple bit rate video streams that varies with video complexity so as to achieve a better visual experience for multiple bit rate streaming. Motion complexity may be considered separately, or along with spatial complexity, in making the resolution decision.

Abstract: A method for detecting presence of a hand gesture in video frames includes receiving video frames having an original resolution, downscaling the received video frames into video frames having a lower resolution, and detecting a motion corresponding to the predefined hand gesture in the downscaled video frames based on temporal motion information in the downscaled video frames. The method also includes detecting a hand shape corresponding to the predefined hand gesture in a candidate search window within one of the downscaled video frames using a binary classifier. The candidate search window corresponds to a motion region containing the detected motion. The method further includes determining whether the received video frames contain the predefined hand gesture based on the hand shape detection.

Abstract: A video encoder uses previously calculated motion information for inter frame coding to achieve faster computation speed for video compression. In a multi bit rate application, motion information produced by motion estimation for inter frame coding of a compressed video bit stream at one bit rate is passed on to a subsequent encoding of the video at a lower bit rate. The video encoder chooses to use the previously calculated motion information for inter frame coding at the lower bit rate if the video resolution is unchanged. A multi core motion information pre-calculation produces motion information prior to encoding by dividing motion estimation of each inter frame to separate CPU cores.

Abstract: In order to extract valid information from video, process video data efficiently, and reduce the transfer stress of network, more and more attention is being paid to the video processing technology. The amount of data in video processing is significantly reduced by using video segmentation and key-frame extraction. So, these two technologies have gradually become the focus of research. With the features of MPEG compressed video stream, a new method is presented for extracting key frames. Firstly, an improved histogram matching method is used for video segmentation. Secondly, the key frames are extracted utilizing the features of I-frame, P-frame and B-frame for each sub-lens. Fidelity and compression ratio are used to measure the validity of the method. Experimental results show that the extracted key frames can summarize the salient content of the video and the method is of good feasibility, high efficiency, and high robustness.

Abstract: A method and apparatus is disclosed herein for motion vector prediction and coding. In one embodiment, the method comprises: deriving N motion vector predictors for a first block that has N motion vectors corresponding to N lists of reference frames and a current frame, including constructing one of the N motion vector predictors when a second block that neighbors the first block and is used for prediction has at least one invalid motion vector, where N is an integer greater than 1; generating N differential motion vectors based on the N motion vectors and N motion vector predictors; and encoding the N differential motion vectors.

Abstract: Digital image and video in their raw form require an enormous amount of storage capacity. Considering the important role played by digital imaging and video in medical and health science, it is necessary to develop a system that produces high degree of compression while preserving critical image/video information. In this paper, we present a hybrid algorithm that performs the discrete cosine transform on the discrete wavelet transform coefficients. Simulation has been carried out on several medical and endoscopic images and videos. The results show that the proposed hybrid algorithm performs much better in term of peak-signal-to-noise-ratio with a higher compression ratio compared to standalone DCT and DWT algorithms. The scheme is intended to be used as the image/video compressor engine in medical imaging and video applications, such as, telemedicine and wireless capsule endoscopy.

Abstract: A method for real-time 2D to 3D video conversion includes receiving a decoded 2D video frame having an original resolution, downscaling the decoded 2D video frame into an associated 2D video frame having a lower resolution, and segmenting objects present in the downscaled 2D video frame into background objects and foreground objects. The method also includes generating a background depth map and a foreground depth map for the downscaled 2D video frame based on the segmented background and foreground objects, and deriving a frame depth map in the original resolution based on the background depth map and the foreground depth map. The method further includes providing a 3D video frame for display at a real-time playback rate. The 3D video frame is generated in the original resolution based on the frame depth map.

Abstract: The present invention provides an improved motion estimation encoder for digital video encoding applications. In one example embodiment, the improved encoder receives a raw image in the form of a current frame and estimates the macroblock motion vector with respect to a reference frame. The encoder then performs an initial local search around an initial motion vector candidate derived from spatio-temporal neighboring macroblock parameters. The encoder then compares the user-defined complexity scalable sum of absolute difference between the original and the associated reference macroblock against an adaptive threshold value for motion estimation convergence. The encoder introduces a global full search around a candidate from a coarser level, in case an initial local search fails. The encoder then selects an inter encoding mode for coding the current macroblock, when the first local search is successful, otherwise the encoder selects the inter or intra encoding mode for encoding the current macroblock by comparing variances of the original and difference macroblocks.

Abstract: This disclosure describes techniques for processing motion vectors such that the resulting motion vectors better correlate with the true motion of a video frame. In one example, the techniques may include comparing a block motion vector corresponding to a video block to a sub-block motion vector corresponding to a sub-block contained within the video block. The techniques may further include selecting one of the block motion vector and the sub-block motion vector as a spatially-estimated motion vector for the sub-block based on the comparison. Motion vectors that better correlate with true motion may be useful in applications such as motion compensated frame interpolation (MCI), moving object tracking, error concealment, or other video post-processing that requires the true motion information.

Abstract: We address an important issue of fully low-cost and low-complexity video encoding for use in resource limited sensors/devices. Conventional distributed video coding (DVC) does not actually meet this requirement because the acquisition of video sequences still relies on the high-cost mechanism (sampling + compression). Recently, we have proposed a distributed compressive video sensing (DCVS) framework to directly capture compressed video data called measurements, while exploiting correlations among successive frames for video reconstruction at the decoder. The core is to integrate the respective characteristics of DVC and compressive sensing (CS) to achieve CS-based single-pixel camera-compatible video encoder. At DCVS decoder, video reconstruction can be formulated as a convex unconstrained optimization problem via solving the sparse coefficients with respect to some basis functions. Nevertheless, the issue of measurement rate allocation has not been considered yet in the literature. Actually, different measurement rates should be adaptively assigned to different local regions by considering the sparsity of each region for improving reconstructed quality. This paper investigates dynamic measurement rate allocation in block-based DCVS, which can adaptively adjust measurement rates by estimating the sparsity of each block via feedback information. Simulation results have indicated the effectiveness of our scheme. It is worth noting that our goal is to develop a novel fully low-complexity video compression paradigm via the emerging compressive sensing and sparse representation technologies, and provide an alternative scheme adaptive to the environment, where raw video data is not available, instead of competing compression performances against the current compression standards (e.g., H.264/AVC) or DVC schemes which need raw data available for encoding.

Abstract: We address an important issue of fully low-cost and low-complex video compression for use in resource-extremely limited sensors/devices. Conventional motion estimation-based video compression or distributed video coding (DVC) techniques all rely on the high-cost mechanism, namely, sensing/sampling and compression are disjointedly performed, resulting in unnecessary consumption of resources. That is, most acquired raw video data will be discarded in the (possibly) complex compression stage. In this paper, we propose a dictionary learning-based distributed compressive video sensing (DCVS) framework to “directly” acquire compressed video data. Embedded in the compressive sensing (CS)-based single-pixel camera architecture, DCVS can compressively sense each video frame in a distributed manner. At DCVS decoder, video reconstruction can be formulated as an l 1 -minimization problem via solving the sparse coefficients with respect to some basis functions. We investigate adaptive dictionary/basis learning for each frame based on the training samples extracted from previous reconstructed neighboring frames and argue that much better basis can be obtained to represent the frame, compared to fixed basis-based representation and recent popular “CS-based DVC” approaches without relying on dictionary learning.

Abstract: To benefit network transmission, the bit stream of the whole frame coded by H.264 is usually grouped into a single packet. However, the error and packet loss during the transmission will result in the distortion of reconstructed video. To deal with this problem, error concealment is widely used to recover the lost frames and to prevent error propagation. In this paper, we propose an accurate motion vector refinement algorithm to further refine the well-known motion copy error concealment method and thus improve the concealed results. By using the motion vector correlation from spatial and temporal domains, a possible motion vector difference variation area has been discovered by the recursive motion vector difference examining approach in the previous frame. Afterwards, a more precise motion vector difference can be derived from the selected motion vector difference variation area. Through the help of our proposed algorithm, the refined motion vectors can be obtained and thus improve the accuracy of motion vectors derived by a motion copy error concealment algorithm. Simulation results show that our proposed algorithm can achieve better performance compared to well-known schemes.

Abstract: High throughput, heavy bandwidth requirement, huge on-chip memory consumption, and complex data flow control are major challenges in high definition integer motion estimation hardware implementation. This paper proposes an efficient very large scale integration architecture for integer multi-resolution motion estimation based on optimized algorithm. There are three major contributions in this paper. First, this paper proposes a hardware friendly multi-resolution motion estimation algorithm well-suited for high definition video encoder. Second, parallel processing element (PE) array structure is proposed to implement three-level hierarchical motion estimation, only 256PEs are enough for one reference frame real-time high definition motion estimation by efficient PE reuse. Third, efficient on-chip reference pixel buffer sharing mechanism between integer and fractional motion estimation is proposed with almost 50% SRAM saving and memory bandwidth reduction. The proposed multi-resolution motion estimation algorithm reached a good balance between complexity and performance with rate distortion optimized variable block size motion estimation support. Also, we have achieved moderate logic circuit and on-chip SRAM consumption. The proposed architecture is well-suited for all MPEG-like video coding standards such as H.264, audio video coding standard, and VC-1.

Abstract: This paper describes a video coding technology proposal submitted by Qualcomm in response to a joint call for proposals (CfP) issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11 (MPEG) in January 2010. The proposed video codec follows a hybrid coding approach based on temporal prediction, followed by transform, quantization, and entropy coding of the residual. Some of its key features are extended block sizes (up to 64 × 64), single pass switched interpolation filters with offsets, mode-dependent directional transforms for intra-coding, luma and chroma high precision filtering, geometric motion partitions, adaptive motion vector resolution and efficient 16-point transforms. It also incorporates internal bit-depth increase and modified quadtree-based adaptive loop filtering. Simulation results are presented to demonstrate the high compression efficiency achieved by the proposed video codec at the expense of moderate increase in encoding and decoding complexity compared to the advanced video coding standard (AVC/H.264). For the random access and low delay configurations, it achieved average bit rate reductions of 30.9% and 33.0% for equivalent peak signal-to-noise ratio, respectively, compared to the corresponding AVC anchors. The proposed codec scored highly in both subjective evaluations and objective metrics and was among the best-performing CfP proposals.

Abstract: We propose a temporal modeling approach for determining image motion from a sequence of images wherein the inherent motion is periodic over time. To exploit the periodic nature of the motion, we use a Fourier harmonic representation to model the temporal evolution of the motion field for the entire sequence. We then determine the motion field simultaneously for the different image frames by estimating the parameters of this representation model, where the model order in the Fourier representation serves as a regularization parameter on the temporal coherence of the motion field. This approach can take advantage of the statistics of all the available data in the image sequence. In our experiments, we tested the proposed approach on several motion types at different noise levels, including translational motion, convergent/divergent motion, and cardiac motion. Our results demonstrate that this approach could lead to more robust estimation of the motion field in the presence of strong imaging noise compared to a frame-by-frame estimation approach.

Abstract: A video encoder may encode video data by adaptively selecting between one-eighth-pixel and one-quarter-pixel precision motion vectors, and signal the selected precision. In one example, an apparatus includes a video encoder to encode a block of video data using a one-eighth-pixel precision motion vector when use of the one-eighth-pixel precision motion vector is determined to be preferable for the block over a one-quarter-pixel precision motion vector, and to generate a signal value indicative of the use of the one-eighth-pixel precision motion vector for the block, and an output interface to output the encoded block and the signal value. A video decoder may be configured to receive the signal value and the encoded block, analyze the signal value to determine whether the block was encoded using one-eighth-pixel precision or one-quarter-pixel precision, and decode the block based on the determination.

Abstract: A transcoder and method of transcoding in which inter-prediction frames of a downsampled video are encoded using downsampled full-resolution residuals in the rate-distortion analysis used to locate a motion vector for a given partition in the downsampled video.

Abstract: Automatic reconstruction of 3D models from video sequences requires selection of appropriate video frames for performing the reconstruction. We introduce a complete method for key frame selection that automatically avoids degeneracies and is robust to inaccurate correspondences caused by motion blur. Our method combines selection criteria based on the number of frame-to-frame point correspondences, Torr’s geometrical robust information criterion (GRIC) scores for the frame-to-frame homography and fundamental matrix, and the point-to-epipolar line cost for the frame-to-frame point correspondence set. In a series of experiments with real and synthetic data sets, we show that our method achieves robust 3D reconstruction in the presence of noise and degenerate motion.

Abstract: A system for producing a motion vector field for a current frame, where the system includes an estimator that selects an initial motion vector from a set of candidate motion vectors and a refiner that refines the initial motion vector.

Abstract: The present disclosure relates to a method and apparatus for improving the encoding efficiency by adaptively changing the resolution of the motion vector in the inter prediction encoding and inter prediction decoding of a video. The apparatus includes: a block identification unit for identifying a colocated block included in a reference picture as a block located at a position equal to a position of a current block; a moving block determiner for determining if the current block is a moving block, based on a motion vector of the colocated block; a motion vector determiner for determining a motion vector of the current block according to a result of the determining of if the current block is a moving block; and a resolution converter for converting a resolution of the motion vector of the colocated block.

Abstract: Video bit rate reduction can be very important for all video streaming application. One of the possible ways to reduce bit rate is decreasing change in time or space domain i.e. changing frame rate or resolution. In this paper we present two no reference metrics mapping frame rate or resolution into MOS. Both models use simple to calculate parameters expressed by sequence spatial and temporal information. The models were estimated and verified upon distinctive video sequence sets. The considered frame rate change varies from 5 to 30 frames per second. The considered resolutions changes from SQCIF to SD.

Abstract: Hybrid video decoder supporting intermediate view synthesis of an intermediate view video from a first- and a second-view video which are predictively coded into a multi-view data signal with frames of the second-view video being spatially subdivided into sub-regions and the multi-view data signal having a prediction mode is provided, having: an extractor configured to respectively extract, from the multi-view data signal, for sub-regions of the frames of the second-view video, a disparity vector and a prediction residual; a predictive reconstructor configured to reconstruct the sub-regions of the frames of the second-view video, by generating a prediction from a reconstructed version of a portion of frames of the first-view video using the disparity vectors and a prediction residual for the respective sub-regions; and an intermediate view synthesizer configured to reconstruct first portions of the intermediate view video.

Abstract: This paper presents a method for estimating vehicle speed by tracking the motion of a vehicle through a sequence of images. The motion is derived using an equation based on spherical projection which relates the image motion to the object motion. Motion tracking is done via the Kanade-Lucas-Tomasi algorithm. The motion equation is reformulated into a dynamical space state model, for which Kalman and Extended Kalman filter are applied to estimate the object velocity as well as to predict the future location of the features. The proposed algorithm is employed on a real-life traffic video captured using an un-calibrated camera to estimate the speed of individual vehicles in the video frames. The main advantages are that it is a simple yet robust method having lower time complexity and less ambiguity in vehicle speed estimations.

Abstract: Medical images often consist of low-contrast objects corrupted by random noise arising in the image acquisition process. Thus, image denoising is one of the fundamental tasks required by medical imaging analysis. In this work, we investigate an adaptive denoising scheme based on the nonlocal (NL)-means algorithm for medical imaging applications. In contrast with the traditional NL-means algorithm, the proposed adaptive NL-means (ANL-means) denoising scheme has three unique features. First, it employs the singular value decomposition (SVD) method and the K-means clustering (K-means) technique for robust classification of blocks in noisy images. Second, the local window is adaptively adjusted to match the local property of a block. Finally, a rotated block matching algorithm is adopted for better similarity matching. Experimental results from both additive white Gaussian noise (AWGN) and Rician noise are given to demonstrate the superior performance of the proposed ANL denoising technique over various image denoising benchmarks in term of both PSNR and perceptual quality comparison.

Abstract: A machine-implemented method of generating a stabilized video sequence includes receiving an input video sequence captured by an image capture device. The input video sequence includes a plurality of pairs of successive frames. Motion sensor data indicative of motion of the image capture device while the input video sequence was being captured is received. A set of matching features for each pair of successive frames is identified. Global motion features are identified in each set of matching features and qualified based on the motion sensor data. The global motion features are indicative of movement of the image capture device. A stabilized video sequence is generated based on the input video sequence and the identified global motion features.

Abstract: Most of the TV manufacturers have released 3DTVs in the summer of 2010 using shutter-glasses technology. 3D video applications are becoming popular in our daily life, especially at home entertainment. Although more and more 3D movies are being made, 3D video contents are still not rich enough to satisfy the future 3D video market. There is a rising demand on new techniques for automatically converting 2D video content to stereoscopic 3D video displays. In this paper, an automatic monoscopic video to stereoscopic 3D video conversion scheme is presented using block-based depth from motion estimation and color segmentation for depth map enhancement. The color based region segmentation provides good region boundary information, which is used to fuse with block-based depth map for eliminating the staircase effect and assigning good depth value in each segmented region. The experimental results show that this scheme can achieve relatively high quality 3D stereoscopic video output.

Abstract: A method for predicting motion vectors to improve compressibility in an image compression codec which processes videos, and an image encoding/decoding apparatus and method using the same. A method for predicting a motion vector used during differential encoding of a motion vector for image encoding, the method including generating a motion vector list with candidate motion vectors for adjacent blocks of a target block, a predictive motion vector of which is to be obtained; calculating each distance between motion vectors included in the motion vector list; and determining a predictive motion vector for the target block by removing motion vectors in order of large distances between the motion vectors.