Stereoscopic Video Coding

Abstract: In the near future, many conventional video applications are likely to be replaced by immersive video to provide a sense of ldquobeing there.rdquo This transition is facilitated by the recent advancement of 3D capture, coding, transmission, and display technologies. Stereoscopic video is the simplest form of 3D video available in the literature. ldquoColor plus depth maprdquo based stereoscopic video has attracted significant attention, as it can reduce storage and bandwidth requirements for the transmission of stereoscopic content over communication channels. However, quality assessment of coded video sequences can currently only be performed reliably using expensive and inconvenient subjective tests. To enable researchers to optimize 3D video systems in a timely fashion, it is essential that reliable objective measures are found. This paper investigates the correlation between subjective and objective evaluation of color plus depth video. The investigation is conducted for different compression ratios, and different video sequences. Transmission over Internet protocol (IP) is also investigated. Subjective tests are performed to determine the image quality and depth perception of a range of differently coded video sequences, with packet loss rates ranging from 0% to 20%. The subjective results are used to determine more accurate objective quality assessment metrics for 3D color plus depth video.

Abstract: In this paper, we present an efficient approach for the compression of stereoscopic video sequences. By extending an H.264/AVC-based multi-view video coding (MVC) codec as described in a previous paper to mixed-resolution imagery, i.e., to stereoscopic videos where the right-eye view had been spatially down-sampled in order to reduce the total amount of data to be encoded, we are able to combine the benefits of both disparity-compensated prediction (DCP) and asymmetric coding. By doing so, we expect to achieve very high compression gains without reducing the overall visual quality of the resulting three-dimensional (3D) percept.

Abstract: Asymmetric stereoscopic video coding takes advantage of the binocular suppression of the human vision by representing one of the views with a lower quality. This paper describes a subjective quality test with asymmetric stereoscopic video. Different options for achieving compressed mixed-quality and mixed-resolution asymmetric stereo video were studied and compared to symmetric stereo video. The bitstreams for different coding arrangements were simulcast-coded according to the Advanced Video Coding (H.264/AVC) standard. The results showed that in most cases, resolution-asymmetric stereo video with the downsampling ratio of 1/2 along both coordinate axes provided similar quality as symmetric and quality-asymmetric full-resolution stereo video. These results were achieved under same bitrate constrain while the processing complexity decreased considerably. Moreover, in all test cases, the symmetric and mixed-quality full-resolution stereoscopic video bitstreams resulted in a similar quality at the same bitrates.

Abstract: Many current as well as emerging applications in areas of entertainment, remote operations, manufacturing industry and medicine can benefit from the depth perception offered by stereoscopic video systems which employ two views of a scene imaged under the constraints imposed by the human visual system. Among the many challenges to be overcome for practical realization and widespread use of 3D/stereoscopic systems are good systems for 3D video capture, display and efficient techniques for digital compression of enormous amounts of data while maintaining compatibility with normal video decoding and display systems. After a brief introduction to the basics of 3D/stereo including issues of depth perception, imaging and display, we present a brief overview of portions of the MPEG-2 video standard that are relevant to our discussion on compression of stereoscopic video. Next, we outline the various approaches for compression of stereoscopic video and then focus on compatible stereoscopic video coding using MPEG-2 Temporal scalability concepts. Compatible coding employing two different types of prediction structures become potentially possible, disparity compensated prediction and combined disparity and motion compensated predictions. To further improve coding performance and display quality, gain and offset preprocessing for reducing mismatch between the two views forming stereoscopic video is considered. We then introduce the various considerations in coding of stereoscopic video at lower bit-rates for the ongoing MPEG-4 standard. A method is proposed that builds on the proven framework of MPEG-2 like coding but introduces additional coding flexibilities to achieve reasonable performance at lower bit-rates for MPEG-4. Next, results of experiments are presented for a variety of combinations of MPEG-2 based coding methods for the left and the right views while employing TV resolution video for a number of sequences and for various bit-rates. The combined disparity and motion-compensated prediction is found to offer the best performance among combinations tested. These results indicate that compression of both views of stereoscopic video of normal TV resolution appears feasible with good quality in a total of 6–8 Mbit/s. Further, results are presented at much lower bit-rates based on the coding method proposed for MPEG-4 on two long test sequences. We then discuss multi-viewpoint video applications, the ongoing efforts towards a multi-viewpoint profile in MPEG-2 and expected direction of multi-viewpoint video coding in MPEG-4.