Abstract: In region-of-interest (ROI)-based video coding, ROI parts of the frame are encoded with higher quality than non-ROI parts. At low bit rates, such encoding may produce attention-grabbing coding artifacts, which may draw viewer's attention away from ROI, thereby degrading visual quality. In this paper, we present a saliency-aware video compression method for ROI-based video coding. The proposed method aims at reducing salient coding artifacts in non-ROI parts of the frame in order to keep user's attention on ROI. Further, the method allows saliency to increase in high quality parts of the frame, and allows saliency to reduce in non-ROI parts. Experimental results indicate that the proposed method is able to improve visual quality of encoded video relative to conventional rate distortion optimized video coding, as well as two state-of-the art perceptual video coding methods.

Abstract: We propose a novel motion model, SteadyFlow, to represent the motion between neighboring video frames for stabilization. A SteadyFlow is a specific optical flow by enforcing strong spatial coherence, such that smoothing feature trajectories can be replaced by smoothing pixel profiles, which are motion vectors collected at the same pixel location in the SteadyFlow over time. In this way, we can avoid brittle feature tracking in a video stabilization system. Besides, SteadyFlow is a more general 2D motion model which can deal with spatially-variant motion. We initialize the SteadyFlow by optical flow and then discard discontinuous motions by a spatial-temporal analysis and fill in missing regions by motion completion. Our experiments demonstrate the effectiveness of our stabilization on real-world challenging videos.

Abstract: We propose a method for detecting insertion and deletion of whole frames in digital videos. We start by strengthening and extending a state of the art method for double encoding detection, and propose a system that is able to locate the point in time where frames have been deleted or inserted, discerning between the two cases. The proposed method is applicable even when different codecs are used for the first and second compression, and performs well even when the second encoding is as strong as the first one.

Abstract: We explore whether we can observe Time's Arrow in a temporal sequence -- is it possible to tell whether a video is running forwards or backwards? We investigate this somewhat philosophical question using computer vision and machine learning techniques. We explore three methods by which we might detect Time's Arrow in video sequences, based on distinct ways in which motion in video sequences might be asymmetric in time. We demonstrate good video forwards/backwards classification results on a selection of YouTube video clips, and on natively-captured sequences (with no temporally-dependent video compression), and examine what motions the models have learned that help discriminate forwards from backwards time.

Abstract: The present invention relates to a method and apparatus for processing a video signal, which can increase the accuracy of the motion vector prediction through motion vector scaling which takes a difference in the temporal distance between reference pictures into consideration. To this end, the present invention provides a video signal processing method and a video signal processing apparatus using the same, and the method comprises the steps of: scaling at least one neighboring partition motion vector for a motion vector prediction of the current partition; scaling the neighboring partition motion vector, which has been selected, when the reference picture of the neighboring partition motion vector is different from the reference picture of the current partition; acquiring a motion vector prediction value of the current partition using the scaled motion vector; and acquiring a motion vector of the current partition using the motion vector prediction value.

Abstract: Techniques coding video data, including a mode for intra prediction of blocks of video data from predictive blocks of video data within the same picture, may include determining a predictive block of video data for the current block of video data, wherein the predictive block of video data is a reconstructed block of video data within the same picture as the current block of video data. A two-dimensional vector, which may be used by a video coder to identify the predictive block of video data, includes a horizontal displacement component and a vertical displacement component relative to the current block of video data. The mode for intra prediction of blocks of video data from predictive blocks of video data within the same picture may be referred to as Intra Block Copy or Intra Motion Compensation.

Abstract: To achieve optimal video quality under bandwidth and power constraints, modern video coding techniques employ lossy coding schemes, which often create compression artifacts that may lead to degradation of perceptual video quality. Understanding and quantifying such perceptual artifacts play important roles in the development of effective video compression, streaming and quality enhancement systems. Moreover, the characteristics of compression artifacts evolve over time due to the continuous adoption of novel coding structures and strategies during the development of new video compression standards. In this paper, we reexamine the perceptual artifacts created by standard video compression, summarizing commonly observed spatial and temporal perceptual distortions in compressed video, with emphasis on the perceptual temporal artifacts that have not been well identified or accounted for in previous studies. Furthermore, a floating effect detection method is proposed that not only detects the existence of floating, but also segments the spatial regions where floating occurs∗.

Abstract: Generally, H.264/AVC video coding standard with hierarchical bipredictive picture (HBP) structure outperforms the classical prediction structures such as “IPPP...” and “IBBP...” through better exploitation of data correlation using reference frames and unequal quantization setting among frames. However, multiple reference frames (MRFs) techniques are not fully exploited in the HBP scheme because of the computational requirement for B-frames, unavailability of adjacent reference frames, and with no explicit sorting of the reference frames for foreground or background being used. To exploit MRFs fully and explicitly in background referencing, we observe that not a single frame of a video is appropriate to be the reference frame as no one covers adequate background of a video. To overcome the problems, we propose a new coding scheme with the HBP, which uses the most common frame in scene (McFIS), generated by background modeling, as a long-term reference (LTR) frame for the third unipredictive reference frame, so that foreground and background areas are expected to be referenced from the two frames in the HBP structure and the McFIS, respectively. There are two approaches to generate McFIS under the proposed methodology. In the first approach, we generate a McFIS using a number of original frames of a scene in a video and then encode it as an I-frame with a higher quality. For the rest of the scene, this generated I-frame is used as an LTR frame. In the second approach, we generate an McFIS from the decoded frames and then use it as an LTR frame, without the need to encode the McFIS. The first and the second approaches are suitable for a video with static background and dynamic background, respectively. In general, the second approach requires more computational time than that of the the first approach. The experiments confirm that the proposed scheme outperforms three state-of-the-art algorithms by improving the image quality significantly with reduced computational time.

Abstract: Automatic video summarization is indispensable for fast browsing and efficient management of large video libraries. In this paper, we introduce an image feature that we refer to as heterogeneity image patch (HIP) index. The proposed HIP index provides a new entropy-based measure of the heterogeneity of patches within any picture. By evaluating this index for every frame in a video sequence, we generate a HIP curve for that sequence. We exploit the HIP curve in solving two categories of video summarization applications: key frame extraction and dynamic video skimming. Under the key frame extraction framework, a set of candidate key frames is selected from abundant video frames based on the HIP curve. Then, a proposed patch-based image dissimilarity measure is used to create affinity matrix of these candidates. Finally, a set of key frames is extracted from the affinity matrix using a min-max based algorithm. Under video skimming, we propose a method to measure the distance between a video and its skimmed representation. The video skimming problem is then mapped into an optimization framework and solved by minimizing a HIP-based distance for a set of extracted excerpts. The HIP framework is pixel-based and does not require semantic information or complex camera motion estimation. Our simulation results are based on experiments performed on consumer videos and are compared with state-of-the-art methods. It is shown that the HIP approach outperforms other leading methods, while maintaining low complexity.

Abstract: Video completion is a computer vision technique to recover the missing values in video sequences by filling the unknown regions with the known information. In recent research, tensor completion, a generalization of matrix completion for higher order data, emerges as a new solution to estimate the missing information in video with the assumption that the video frames are homogenous and correlated. However, each video clip often stores the heterogeneous episodes and the correlations among all video frames are not high. Thus, the regular tenor completion methods are not suitable to recover the video missing values in practical applications. To solve this problem, we propose a novel spatially-temporally consistent tensor completion method for recovering the video missing data. Instead of minimizing the average of the trace norms of all matrices unfolded along each mode of a tensor data, we introduce a new smoothness regularization along video time direction to utilize the temporal information between consecutive video frames. Meanwhile, we also minimize the trace norm of each individual video frame to employ the spatial correlations among pixels. Different to previous tensor completion approaches, our new method can keep the spatio-temporal consistency in video and do not assume the global correlation in video frames. Thus, the proposed method can be applied to the general and practical video completion applications. Our method shows promising results in all evaluations on both 3D biomedical image sequence and video benchmark data sets.

Abstract: The present paper proposes a novel multiple-LEDarrays acquisition for an infrastructure-to-vehicle visible light communication (I2V-VLC) using LED arrays (transmitter) and an in-vehicle high-speed image sensor (receiver). In order to achieve a robust detection of LED arrays, we employ the block matching algorithm, which is a way of finding a corresponding position between two successive frames. The proposed method divides a captured image into a number of small domains (blocks) and determines if the LED array is present or absent using the block matching. We perform I2V-VLC experiments with multiple-LED arrays and evaluate the acquisition capability of the proposed method.

Abstract: Multiple video picture frames are combined into a spatial multiplex video picture frame that may be fully decoded and displayed The video display of the spatial multiplex video picture frame is a composite combination of all of the video picture frames that have been combined, and may have an appearance such as a mosaic Multiplexing the video picture frames involves removing picture headers, creating a picture header for the spatial multiplex video picture frame, and altering the headers of individual components of each video picture frame The new header for the spatial multiplex video picture frame indicates a slice format frame, and headers of the individual components are altered to provide a slice format based picture position for each video picture frame The headers of the individual components are altered to become slice based, such as in accordance with the ITU-T H263 video standard, prior to establishing the slice based picture position if the frames are not already of the slice format

Abstract: Motion estimation is one of the major problems in developing video coding applications. Among all motion estimation approaches, Block matching (BM) algorithms are the most popular methods due to their effectiveness and simplicity for both software and hardware implementations. A BM approach assumes that the movement of pixels within a defined region of the current frame (Macro-Block, MB) can be modeled as a translation of pixels contained in the previous frame. In this procedure, the motion vector is obtained by minimizing the sum of absolute differences (SAD) produced by the MB of the current frame over a determined search window from the previous frame. The SAD evaluation is computationally expensive and represents the most consuming operation in the BM process. The most straightforward BM method is the full search algorithm (FSA) which finds the most accurate motion vector, calculating exhaustively the SAD values for all elements of the search window. Over this decade, several fast BM algorithms have been proposed to reduce the number of SAD operations by calculating only a fixed subset of search locations at the price of a poor accuracy. In this paper, a new algorithm based on Differential Evolution (DE) is proposed to reduce the number of search locations in the BM process. In order to avoid computing several search locations, the algorithm estimates the SAD values (fitness) for some locations using the SAD values of previously calculated neighboring positions. Since the proposed algorithm does not consider any fixed search pattern or other different assumption, a high probability for finding the true minimum (accurate motion vector) is expected. In comparison to other fast BM algorithms, the proposed method deploys more accurate motion vectors yet delivering competitive time rates.

Abstract: In digital forensics, recovery of a damaged or altered video file plays a crucial role in searching for evidences to resolve a criminal case. This paper presents a frame-based recovery technique of a corrupted video file using the specifications of a codec used to encode the video data. A video frame is the minimum meaningful unit of video data. Many existing approaches attempt to recover a video file using file structure rather than frame structure. In case a target video file is severely fragmented or even has a portion of video overwritten by other video content, however, video file recovery of existing approaches may fail. The proposed approach addresses how to extract video frames from a portion of video to be restored as well as how to connect extracted video frames together according to the codec specifications. Experiment results show that the proposed technique successfully restores fragmented video files regardless of the amount of fragmentations. For a corrupted video file containing overwritten segments, the proposed technique can recover most of the video content in non-overwritten segments of the video file.

Abstract: Various embodiments each include at least one of systems, methods, devices, and software for environment mapping with automatic motion model selection. One embodiment in the form of a method includes receiving a video frame captured by a camera device into memory and estimating a type of motion from a previously received video frame held in memory to the received video frame. When the type of motion is the same as motion type of a current keyframe group held in memory, the method includes adding the received video frame to the current keyframe group. Conversely, when the type of motion is not the same motion type of the current keyframe group held in memory, the method includes creating a new keyframe group in memory and adding the received video frame to the new keyframe group.

Abstract: This paper proposes an error concealment technique for video transmission, focusing on motion vector (MV) recovery for both inter- and intra-coded frames, to improve video quality at decoder when video bit stream data incur transmission errors. The proposed algorithm considers slice (i.e., a row of macroblocks (MBs)) errors and uses DP (Dynamic Programming) optimization technique to estimate the lost MVs in a global manner, differing from the traditional Boundary Matching Algorithm (BMA) and others that recover MVs independently for individual MBs in an erroneous slice. We also propose an iterative DP process based on 8 × 8 pixels blocks to resolve finer motions (for 8 × 8, 8 × 16, and 16 × 8 pixels blocks) that will aid in the enhancement of reconstruction quality. Experiment results show that our algorithm outperforms the well-known BMA by up to 7.28 dB and the DMVE and another prior work by Qian by up to 1.0 dB at a packet loss rate of 15%. Subjective evaluation shows that our algorithm is especially promising in preserving line/curve features and motion details.

Abstract: Data streams from multiple image sensors may be combined in order to form, for example, an interleaved video stream. The video stream may be encoded using a motion estimation encoder. Output of the video encoder may be processed (e.g., parsed) in order to extract motion information present in the encoded video. The motion information may be utilized in order to determine a depth of visual scene, such as by using binocular disparity between two or more images by an adaptive controller in order to detect one or more objects salient to a given task. In one variant, depth information is utilized during control and operation of mobile robotic devices.

Abstract: This Article Presents The Design Of Continuous Video Streaming Compression of A Networked System For Joint Compression, Error Detection And For Error Correction. Video Compression Applications Are Becoming More Popular Over Wireless Multimedia Sensor Networks. To Save Energy In Bit Transmissions And Receptions Over A Video Sensor Network, The Video File Contents Need To Be Encoded Packet By Packet Before Its Transmission Over Network. The Quality At The Receiver Side Can Be Maximized By Designing The Cross Layer System. The Transmission Rate And Encoding Rate Can Be Controlled By The Cross Layer System. The Fairness Of The Received Video Quality Can Be Developed By The Rate Controller. Instead, An Adaptive Parity Scheme That Drops Samples In Error Is Proposed And Shown To Improve Video Quality. Finally A Low Complexity, Adaptive Video Encoder, Is Proposed That Performs Low Complexity Motion Estimation On Sensors And Implemented To Further Optimize Energy Consumption. Thus Greatly Reducing The Amount Of Data To Be Transmitted And Video File Can Be Decompressed.

Abstract: In this paper, we propose a nonparametric camera motion descriptor for video shot classification. In the proposed method, a motion vector field (MVF) is constructed for each consecutive video frame by computing the motion vector (MV) of each macroblock. Then, the MVFs are divided into a number of local region of equal size. Next, the inconsistent/noisy MVs of each local region are eliminated by a motion consistency analysis. The remaining MVs of each local region from a number of consecutive frames are further collected for a compact representation. Initially, a matrix is formed using the MVs. Then, the matrix is decomposed using a singular value decomposition technique to represent the dominant motion. Finally, the angle of the most variance retaining principal component is computed and quantized to represent the motion of a local region by using a histogram. In order to represent the global camera motion, the local histograms are combined. The effectiveness of the proposed motion descriptor for video shot classification is tested by using a support vector machine. First, the proposed camera motion descriptors for video shots classification are computed on a video data set consisting of regular camera motion patterns (e. g., pan, zoom, tilt, static). Then, we apply the camera motion descriptors with an extended set of features to the classification of cinematographic shots. The experimental results show that the proposed shot level camera motion descriptor has a strong discriminative capability to classify different camera motion patterns of different videos effectively. We also show that our approach outperforms state-of-the-art methods.

Abstract: Detection of frame deletion is of great significance in the field of video forensics. Several approaches have been presented through analyzing the side effect caused by frame deletion. However, most of the current approaches can detect the existence of frame deletion but not the exact location of it. In this paper, we present a method which can directly locate the frame deletion point. Through the analysis of the distinguishing fluctuation feature of motion residual caused by frame deletion compared to interference frames and ordinary video content jitter in tampered video sequence, an algorithm based on the total motion residual of video frame is proposed to detect the frame deletion point. Moreover, an initiative processing procedure for frame motion residual and an adaptive threshold detector are introduced so that the robustness of the detection can be markedly improved. Experimental results show that the proposed algorithm is effective in generalized scenarios such as different encoding settings, rapid or slow motion sequences and multiple group of picture deletion. It also has a high performance that the true positive rate reaches 90% and the false alarm rate is less than 0.8%.

Abstract: A method for detecting and tracking a target object is described. The method includes performing motion-based tracking for a current video frame by comparing a previous video frame and the current video frame. The method also includes selectively performing object detection in the current video frame based on a tracked parameter.

Abstract: Video stabilization is an important technique in present day digital cameras as most of the cameras are hand-held, mounted on moving platforms or subjected to atmospheric vibrations. In this paper we propose a novel video stabilization scheme based on estimating the camera motion using maximally stable extremal region features. These features traditionally used in wide baseline stereo problems were never explored for video stabilization purposes. Through our extensive experiments show we how some properties of these region features are suitable for the stabilization task. After estimating the global camera motion parameters using these region features, we smooth the motion parameters using a gaussian filter to retain the desired motion. Finally, motion compensation is carried out to obtain a stabilized video sequence. A number of examples on real and synthetic videos demonstrate the effectiveness of our proposed approach. We compare our results to existing techniques and show how our proposed approach compares favorably to them. Interframe Transformation Fidelity is used for objective evaluation of our proposed approach.

Abstract: Techniques related to dominant motion compensated prediction for next generation video coding are described.

Abstract: With the advancement in technology, it becomes easy for some individuals to use digital data without the permission of the owner. To increase authentication and security of copyrights, digital watermarking was introduced. Medical videos contain very significant information about the condition of the patient. A good watermarking scheme should always contain very less distortion. Videos generally contain huge temporal redundancy, which demands the use of motion vector estimation technique in order to remove the temporal redundancy. In this work, an image was watermarked inside the motion vector of two consecutive frames of an echo-cardiograph video. Quality analysis of the recovered video with the original video proves the robustness of the proposed scheme.

Abstract: With the increasing volume of video data, how to analyze and browse video in a fast and effective way has become an urgent problem in applications. This letter proposes a novel video synopsis method in compressed domain for browsing video captured by static cameras. Synopsis video is a video abstraction, which displays moving objects from different periods simultaneously on the primary background contents of original video. To overcome the low efficiency of traditional video synopsis for compressed video, our method presents a new graph cut algorithm to extract objects tubes and meanwhile gives a fast solution to minimize energy function in compressed domain. Experimental results in H.264 video have demonstrated the high-efficiency of this new video synopsis scheme for massive video browsing.

Abstract: A post-processing technique is introduced to improve the Eulerian video magnification method, which is a state-of-the-art motion magnification method to manipulate small movements in videos based on spatio-temporal filtering. The proposed method use the Eulerian video magnification as a video spatio-temporal motion analyzer to get the pixel-level motion mapping. Then the input video pixels are warped based-on this mapping to amplify the motion. This processing does not involve pixel value modifying, which makes it supports larger amplification and is significantly less influenced by the frame noise.

Abstract: Based on the classical fractal video compression method, an improved object-based stereo video compression scheme with Shape-Adaptive DCT is proposed in this paper. Firstly, we use more effective macroblock partition scheme instead of classical quadtree partition scheme; thus reducing the block searching strategy. The stereo fractal video coding is proposed which matches the macroblock with two reference frames in left and right view results in increasing compression ratio and reducing bit rate when transmitting compressed stereo data. The stereo codec combines the Motion Compensation Prediction (MCP) and Disparity Compensation Prediction (DCP). Fractal coding is adopted and each object is encoded independently by a prior video segmentation alpha plane, which is defined exactly as in MPEG-4. The testing results with the nature monocular and stereo video sequences provide promising performances at low bit rate coding. We believe it will be a powerful and efficient technique for the object-based monocular and stereo video sequences coding.