Visual Content Retargeting: Algorithms, Applications, and Quality Assessment

Video retargeting from a full-resolution video to a reduced-resolution display will inevitably cause information loss. Content-aware retargeting has proven to be an efficient means to avoid critical visual information loss while resizing an image/video. Nevertheless, retargeting a 2D/3D video can often lead to visually annoying distortions in object shape, scene depth, and temporal coherency. Preserving the shape and depth information as well as maintaining the spatio-temporal coherency of a retargeted video is very critical on visual quality. In this tutorial, we will first introduce the technical challenges and recent advances in contentaware retargeting. Then we will show how to use a panoramic mosaic to guide the scaling of corresponding regions of video frames in a video shot to ensure good spatio-temporal coherency. We will also introduce our recent results on simultaneously preserving scene depths, object shapes, and temporal coherency in stereoscopic video retargeting based on constraints relaxation. Then, we will present objective quality metrics based on geometric distortion and information loss for automatically evaluating the visual quality of a retargeted image and how to incorporate the metrics in video retargeting to improve visual quality. Finally, we will show how to construct a scalable video coder which supports content-adaptive spatial scalability (e.g., the base-layer and enhancement-layer videos are of different resolution and different aspect ratios) with good coding efficiency.