Abstract: A motion estimation and compensation technique is provided for interlaced digital video such as video object planes (VOPs). Predictor motion vectors for use in differentially encoding a current field coded macroblock are obtained using the median of motion vectors of surrounding blocks or macroblocks. When a surrounding macroblock is itself interlaced coded, an average motion vector for that macroblock is used, with fractional pixel values being mapped to the half-pixel. When the current block is not interlaced coded but a surrounding block is, the field motion vectors may be used individually or averaged. In a repetitive padding technique for an interlaced coded VOP, the even and odd lines of the VOP and surrounding block are grouped. Within each field, exterior pixels are padded by setting them to the value of the nearest boundary pixel, or to an average of two boundary pixels. The lines are then reordered to provide a single padded reference VOP image.

Abstract: A system for coding of digital video images such as bi-directionally predicted video object planes (B-VOPs), in particular, where the B-VOP and/or a reference image used to code the B-VOP is interlaced coded. For a B-VOP macroblock which is co-sited with a field predicted macroblock of a future anchor picture, direct mode prediction is made by calculating four field motion vectors, then generating the prediction macroblock. The four field motion vectors and their reference fields are determined from (1) an offset term of the current macroblock's coding vector, (2) the two future anchor picture field motion vectors, (3) the reference field used by the two field motion vectors of the co-sited future anchor macroblock, and (4) the temporal spacing, in field periods, between the current B-VOP fields and the anchor fields. Additionally, a coding mode decision process for the current MB selects a forward, backward, or average field coding mode according to a minimum sum of absolute differences (SAD) error which is obtained over the top and bottom fields of the current MB.

Abstract: In this paper, we present a new digital image stabilization (DIS) scheme based on bit-plane matching (BPM) The proposed DIS system performs motion estimation using 1-bit planes which are extracted from a video sequence This motion estimation technique can be realized using only Boolean functions which have significantly reduced computational complexity, while the accuracy of motion estimation is maintained In the second part of this paper, a median-based motion correction scheme is proposed which is robust to various irregular conditions such as moving objects and intentional panning Simulation results show that the proposed DIS algorithm exhibits better performance compared with existing other algorithms when applied to real video signals

Abstract: Temporal and spatial scaling of video images including video object planes (VOPs) in an input digital video sequence is provided. Coding efficiency is improved by adaptively compressing scaled field mode video. Upsampled VOPs in the enhancement layer are reordered to provide a greater correlation with the input video sequence based on a linear criteria. The resulting residue is coded using a spatial transformation such as the DCT. A motion compensation scheme is used for coding enhancement layer VOPs by scaling motion vectors which have already been determined for the base layer VOPs. A reduced search area whose center is defined by the scaled motion vectors is provided. The motion compensation scheme is suitable for use with scaled frame mode or field mode video. Various processor configurations achieve particular scaleable coding results. Applications of scaleable coding include stereoscopic video, picture-in-picture, preview access channels, and ATM communications.

Abstract: In a method of estimating motion, at least two motion parameter sets are generated (PE1-PEn) from input video data (n, n-1), a motion parameter set being a set of parameters describing motion in an image, by means of which motion parameter set motion vectors can be calculated. One motion parameter set indicates a zero velocity for all image parts in an image, and each motion parameter set has corresponding local match errors. Output motion data are determined from the input video data (n, n-1) in dependence on the at least two motion parameter sets, wherein the importance of each motion parameter set in calculating the output motion data depends on the motion parameter sets' local match errors.

Abstract: Multimedia data security is important for multimedia commerce. Previous cryptography studies have focused on text data. The encryption algorithms developed to secure text data may not be suitable to multimedia applications because of large data sizes and real time constraints. For multimedia applications, light weight encryption algorithms are attractive. We present an efficient MPEG video encryption algorithm. This algorithm uses a secret key randomly changing the sign bits of encoded differential values of DC coefficients of I pictures and the sign bits of encoded differential values of motion vectors of B and P pictures. The encryption effects are achieved by the IDCT during MPEG video decompression processing. This algorithm adds very small overhead to MPEG codec. A software implementation is fast enough to meet the real time requirement of MPEG video applications. Experimental results show that this algorithm achieves satisfying results. We believe that it can be used to secure video-on-demand, video conferencing and video email applications.

Abstract: By exploiting the correlation of ultrasound speckle patterns that result from scattering by underlying tissue elements, two-dimensional tissue motion theoretically can be recovered by tracking the apparent movement of the associated speckle patterns. Speckle tracking, however, is an ill-posed inverse problem because of temporal decorrelation of the speckle patterns and the inherent low signal-to-noise ratio of medical ultrasonic images. This paper investigates the use of an adaptive deformable mesh for nonrigid tissue motion recovery from ultrasound images. The nodes connecting the mesh elements are allocated adaptively to stable speckle patterns that are less susceptible to temporal decorrelation. The authors use the approach of finite clement analysis in manipulating the irregular mesh elements. A novel deformable block matching algorithm, making use of a Lagrange element for higher-order description of local motion, is proposed to estimate a nonrigid motion vector at each node. In order to ensure that the motion estimates are admissible to a physically plausible solution, the nodal displacements are regularized by minimizing the strain energy associated with the mesh deformations. Experiments based on ultrasound images of a tissue mimicking phantom and a muscle undergoing contraction, and on computer simulations, have shown that the proposed algorithm can successfully track nonrigid displacement fields.

Abstract: Typical video search is based on queries involving a single shot. We generalize this problem by allowing queries that involve a video clip. We propose two schemes for query by video clip. In the first scheme, retrieval based on key frames where the database and query video are segmented into shots which are represented via key frames. For every query key frame, a similarity value (using color, texture, and motion) is associated with the key frames in the database video clip. Boundaries marking highly similar consecutive shots are then used to generate the set of retrieved video sub-clips. In the second scheme, in retrieval using sub-sampled frames, we uniformly sub-sample the query clip as well as the database video. The retrieval is based on matching color and texture features of the sub-sampled frames. Initial experiments on two video databases show promising results.

Abstract: Synthesizing of video frames that have been dropped by a video encoder is achieved by interpolating between decoded frames at a decoder. The method consists of successive refinement stages that increase in computational complexity. Starting with a spatio-temporal median filtering approach, each stage uses information that improves the quality of the interpolated frames, such as bit stream motion information, decoder-based motion estimation and motion-based state segmentation of regions. By using more computational resources, each of these stages results in an improved quality of interpolated video. The motion compensation techniques are based on block-based motion estimation of the kind used by block-transform based video encoders. More accurate motion estimates are obtained by using a combination of forward and backward block motion estimation. The method is further extended by incorporating global/local motion estimation based on the segmentation information, and employing image warping techniques to compensate for motion resulting from deformations.

Abstract: An adaptive region-based, multi-scale, motion compensated video compression algorithm design for transmission over hostile communication channels. The algorithm is embodied in a video encoder that extracts spatial information from video frames to create video regions that are then decomposed into sub-bands of different perceptual importance before being compressed and transmitted independently. The system further uses unequal error protection, prioritized transmission and reconstruction to guarantee a minimum spatial and temporal resolution at the receiver. In particular, the region segmented frames bound both spatial and temporal error propagation within frames. A connection-level inter-region statistical multiplexing scheme is also employed to ensure optimal utilization of reserved transmission bandwidth.

Abstract: This paper investigates the dynamic frame skipping strategy in video transcoding. To speed up the operation, a video transcoder usually reuses the decoded motion vectors to reencode the video sequences at a lower bit-rate. When frame skipping is allowed in a transcoder, those motion vectors can not be reused because the motion vectors of the current frame is no longer estimated from the immediate past frame. To reduce the computational complexity of motion vectors reestimation, a bilinear interpolation approach is developed to overcome this problem. Based on these interpolated motion vectors, the search range can be much reduced. Furthermore, we propose a frame rate control scheme which can dynamically adjust the number of skipped frames according to the accumulated magnitude of the motion vectors. As a result, the decoded sequence can present much smoother motion.

Abstract: The paper addresses automatic scene change detection, key frame selection, and similarity ranking which constitute the main steps of a content based video abstraction system. Unlike other methods, the proposed algorithm performs scene change detection and key frame selection in one step. We treat scene change detection as a two-class classification problem and employ automatic threshold selection techniques originally developed for image binarization. A quantitative measure for retrieval of similar scenes according to their color content is also defined. The described scheme can be applied to both uncompressed and MPEG compressed video, and can be implemented in real time. Performance of the algorithm has been analyzed on real TV sequences, and comparison with some previously introduced techniques are provided.

Abstract: Amateur digital video footage, shot with a hand-held camera, is characterized by visually annoying, jerky movement between successive frames. In this paper a fast and robust image stabilization technique is presented to remove the effects of unwanted camera motion while not effecting true pans or zooms. Unlike other image stabilization techniques, which are typically used as a pre-processing step to more complicated image processing operations, the proposed technique is specifically aimed at providing "visually jerk-free video" for consumer viewing. Empirical evidence suggested that a simple global translational model for each pair of successive images followed by local integration of cumulative motion is sufficient for this purpose. We use a novel technique to estimate the translational motion model which reduces the total computational burden to less than 1 computation/pixel, while providing robust jerk elimination, thus making real-time operation possible on most PC/UNIX platforms. Tests with a number of video sequences have shown the efficacy of the proposed method.

Abstract: A system encodes video by detecting the location of a facial region of a frame of the video. Sensitivity information is calculated for each of a plurality of locations within the video based upon the location of the facial region. The frame is encoded in manner that provides a substantially uniform apparent quality of the plurality of locations to the viewer when the viewer is observing the facial region of the video.

Abstract: A method and system for object-based video retrieval and indexing include a configuration detection processor for deriving quantitative attribute information for video frames in a compressed video stream. The quantitative attribute information includes object data for a video frame, including the number of objects and their orientation within the video frame and the size, shape, texture, and motion of each object. A configuration comparison processor compares object data from first and second frames to determine differences between first frame video objects and second frame video objects. The configuration comparison processor has a shot boundary detection mode in which it cooperates with a shot boundary detector to identify shot boundaries within a video sequence. In a key frame selection mode, the configuration comparison processor cooperates with a key frame selector to select key frames from the video sequence. A key instance selector communicates with the configuration comparison processor during a key instance selection mode to select key instances of video objects based on differences between first and second instances of video objects. The configuration comparison processor cooperates with a camera operation detector to identify camera operations such as zoom, tracking, and panning within the video sequence. A special effects detector cooperates with the configuration comparison processor to detect special effects video edits such as wipe, dissolve, and fade. The configuration comparison processor and a query match detector enable a user to configure object-based queries and to retrieve video sequences or video frames which include a query video object.

Abstract: Shape information coding device for interlaced scanning video and method in which the shape information coding device and method can detect an amount of motion of object video, on coding of interlaced scanning video, select field or frame coding mode in accordance with the detected result, perform motion compensation in a field unit if the selected coding mode is the field mode, and perform motion compensation in a frame unit if the selected coding mode is the frame mode. In addition, the present invention can construct one frame with two fields, upon coding of shape information for the interlaced scanning video, and then determine a motion vector predictor for shape by using motion information of adjacent block so as to perform an effective coding of the shape information motion information. At this time, the coding efficiency can be improved by a method contemplating preceding a motion vector having the same motion prediction mode as the current motion vector for the motion vector predictor for shape having a high similarity and determining the motion vector predictor for shape and by a method contemplating performing coding of the field block type information and the filed discrimination information for one field of the two fields.

Abstract: Traditionally, the re-use of motion vectors extracted from incoming video bit-stream during transcoding has been widely accepted. However, this simple re-use scheme introduces significant quality degradation in many applications including the situation when the frame-rate conversion is needed. In this paper, we analyzed the quantization errors that cause the extracted motion vectors to be non-optimal and we performed simulations to show the quality degradation due to the inaccurate motion vectors during transcoding. To improve the video quality, we proposed an adaptive motion vector refinement. With a highly reduced computational complexity, the proposed adaptive motion vector refinement achieves significant quality improvement in comparison to the conventional motion vector re-use scheme. In addition, the adaptive motion vector refinement is almost as good as performing a new full-scale motion estimation.

Abstract: A method of generating a 2-D extended image from a video sequence representing a natural 3-D scene first determines motion parameters for a camera that recorded the scene with respect to a bakcground object from the video sequence using a structure-from-motion algorithm. The motion parameters include a rotation matrix, a translation vector and a depth map representing the depth of each point in the background object from the camera. Next from the motion parameters and depth map the 2-D extended image is generated for the background object as a composition of the images from the video sequence using a plane perspective projection technique. The background object may be layered as a function of depth and flatness criteria to form a set of layered 2-D extended images for the background object from the video sequence.

Abstract: Motion estimation and compensation is widely used for exploiting temporal correlation within an image sequence. To find motion vectors that lead to high compression, most motion estimation approaches use a source distortion measure, such as mean-square error (MSE) or mean-absolute error (MAE), as a search criterion. When incorporated into a closed-loop motion compensated (MC) transform video coder, these schemes produce noisy motion fields which significantly increase the bit-rates required to represent motion vectors. In view of this problem, this paper presents a rate-distortion optimal motion estimation algorithm. The proposed scheme improves rate performance of the estimated motion field while maintaining the peak signal-to-noise ratio (PSNR) prediction quality of the distortion-based methods, thereby enabling an efficient bit allocation between motion information and transform-coded prediction residuals. For coders in which motion vectors are differentially encoded, the rate-distortion optimization process is formulated as a shortest-path-finding problem. Adopting this framework, we show that the optimal solution for the conventional block-based motion estimation, followed by one-dimensional (1-D) differential coding and Huffman coding, can be obtained by using dynamic programming or the Viterbi algorithm. We propose an effective fast algorithm that closely approximates the optimal performance while requiring considerably less complexity. Our experimental results demonstrate overall gains in the range of 0.3-1.5 dB.

Abstract: Fast motion estimation algorithms for compression of moving pictures are widely required for real-time video encoding instead of the full search (FS) block matching algorithm. It provides optimal error performance but requires enormous computation for calculating the motion vector. In many fast algorithms, the three-step search (TSS) has been used for real time video encoding and low bit-rate video communications because of the reduced computations, simplicity and reasonable performance. Other modified algorithms of the TSS have been studied for the speed of computation and the error performance for motion estimation. This paper proposes a new algorithm (fast three-step search) for further reduction in computational complexity using a unimodal error surface assumption (UESA) without serious degradation of the error performance. We show that the computational complexity is further reduced with additional subsampling of the matching block. It is also shown that the proposed algorithm (FTSS) is computationally efficient while keeping the same performance as that of the TSS.

Abstract: The invention relates to a filtering method used for a video signal at the receiver. It is mostly suited for video compression algorithms utilizing DCT-based video compression technology. In the invention the boundaries ( 49 ) between adjacent video blocks (B 5 and B 6 ) are filtered based on the amount of activity inside the adjacent video blocks and the activity at the boundary between the adjacent video blocks. If the filtering according to the invention is performed, it is focused to a certain number of bits ( 42, 43, 44, 45, 46, 47 ) close to the boundary ( 49 ). The filtering is done by adjusting the numerical values of each video pixel close to the boundary towards a reference line, which is defined as a linear equation leading from the numerical value of a first reference pixel ( 41 ) to the numerical value of a second reference pixel ( 48 ). The reference pixels ( 41, 48 ) are selected from the adjacent video blocks to present the smooth movement over the boundary between the adjacent video blocks (B 5 and B 6 ).

Abstract: A sprite generation method used in video coding generates a sprite from the video objects in the frames of a video sequence. The method estimates the motion (1200) between a video object in a current frame and a sprite constructed from video objects for previous frames. Specifically, the method computes motion coefficients of a 2D transform that minimizes the intensity errors between pixels in the video object and corresponding pixels inside the sprite. The method uses the motion coefficients from the previous frame (1206) as a starting point to minimizing the intensity errors. After estimating the motion parameters for an object in the current frame, the method transforms the video object to the coordinate system of the sprite. The method blends (1204) the warped pixels (1202) of the video object with the pixels at corresponding positions in the sprite using rounding average such that each video object in the video sequence provides substantially the same contribution to the sprite.

Abstract: In this work we apply a direct sequence spread spectrum model to the watermarking of digital video. First, the video signal is modeled as a sequence of bit planes arranged along the time axis. Watermarking of this sequence is a two layer operation. A controlling m-sequence first establishes a pseudorandom order in the bitplane stream for later watermarking. Watermarks, defined as m-frames, supplant the tagged bitplanes. We have shown that the watermark, when limited to the 4 lowest bitplanes of an 8-bit video, are unnoticeable. Moreover, attempts in corrupting the image to destroy the watermark renders the video useless before damaging the seal itself. The watermarked video is also robust to video editing attempts such as subsampling, frame reordering etc. The watermark is also identifiable from very short segments of video. Individual frames extracted from the video will also contain copyright information.

Abstract: MPEG compression for video/audio data is completed in real- or better than real-time by using computer processors in thousands of television receiver set-top boxes interconnected to a cable television network. The set-top boxes form a massively parallel, distributed computer network. MPEG compression for video/audio services is completed by partitioning the processing of video/audio frames into subtasks, and distributing these subtasks to set-top boxes that are not being used for subscriber services. After MPEG processing, the compressed video/audio frames are collected from the set-top boxes and are reconstructed into a compressed video/audio stream in the proper temporal order. If desired, the original video data can be JPEG-encoded prior to distribution to the individual set-top boxes, and then JPEG-decoded at the boxes. So doing reduces bandwidth constraints on the distribution network. Moreover, the lossy JPEG encoding/decoding process removes high frequency components from the original video data, facilitating and speeding the MPEG compression.

Abstract: This paper presents a novel approach to stabilize video sequences digitally. A 2.5D inter-frame motion model is proposed so that the stabilization system can work in situations where significant depth changes are present and the camera has both rotational and translational movements. An inertial model for motion filtering is proposed in order to eliminate the vibration of the video sequences and to achieve good perceptual properties. The implementation of this new approach integrates four modules: pyramid-based motion detection, motion identification and 2.5D motion parameter estimation, inertial motion filtering, and affine-based motion compensation. The stabilization system can smooth unwanted vibrations of video sequences in real-time. We test the system on IBM PC compatible machines and the experimental results show that our algorithm outperforms many algorithms that require parallel pipeline image processing machines.

Abstract: This paper describes a new motion estimation algorithm that is suitable for hardware implementation and substantially reduces the hardware cost by using a low bit-resolution image in the block matching. In the low bit-resolution image generation, adaptive quantization is employed to reduce the bit resolution of the pixel values, which is better than simple truncation of the least significant bits in preserving the dynamic range of the pixel values. The proposed algorithm consists of two search steps: in the low-resolution search, a set of candidate motion vectors is determined, and in the full-resolution search, the motion vector is found from these candidate motion vectors. The hardware cost of the proposed algorithm is 1/17 times of the full search algorithm, while its peak signal-to-noise ratio is better than that of the 4:1 alternate subsampling for the search range of /spl plusmn/32/spl times//spl plusmn/32. A VLSI architecture of the proposed algorithm is also described, which can concurrently perform two prediction modes of the MPEG2 video standard with the search range of (-32.0,-32.0)-(+31.5,+31.5). We fabricated a MPEG2 motion estimator with a 0.5-/spl mu/m triple-metal CMOS technology. The VLSI chip includes 110 K gates of random logic and 90 K bits of SRAM in a die size of 11.5 mm/spl times/12.5 mm. The full functionality of the fabricated chip was confirmed with an MPEG2 encoder chip.

Abstract: A lightweight genetic search algorithm (LGSA) is proposed. Different evolution schemes are investigated, such that the control overheads are largely reduced. It is also shown that the proposed LGSA can be viewed as a novel expansion of the three-step search algorithm (TSS). It can be seen from the simulation results that the performance of LGSA is very similar to that of the full search algorithm (FSA), and the computational complexity is much lower than that of FSA and other previously proposed genetic motion estimation algorithms.