KEYWORDS: Particles, Video compression, Motion models, Video, Cameras, Head, 3D video compression, Quantization, Detection and tracking algorithms, Telecommunications
In immersive communication applications, knowing the user's viewing position can help improve the efficiency of
multiview compression and streaming significantly, since often only a subset of the views are needed to synthesize the
desired view(s). However, uncertainty regarding the viewer location can have negative impacts on the rendering quality.
In this paper, we propose an algorithm to improve the robustness of view-dependent compression schemes by jointly
performing user tracking and compression. A face tracker tracks the user's head location and sends the probability
distribution of the face locations as one or many particles. The server then applies motion model to the particles and
compresses the multiview video accordingly in order to improve the expected rendering quality of the viewer.
Experimental results show significantly improved robustness against tracking errors.
In this paper we study the non-uniform sampling problem for image-based rendering (IBR). We first propose a general position-interval error (PIE) function that can help solve practical sampling problems. We then propose two approaches to capturing IBR data non-uniformly based on PIE, namely progressive capturing (PCAP) and rearranged capturing (RCAP). PCAP is applicable for static scenes. Based on the images captured so far, PCAP determines where to take the next image. RCAP, on the other hand, is for capturing both static and dynamic scenes. The goal is to intelligently arrange the positions of a limited number of cameras such that the final rendering quality is optimal. Experimental results demonstrate that the non-uniform sampling approaches outperform the traditional uniform methods.
In this paper, we describe a system for wandering in a realistic environment over the Internet. The environment is captured by the concentric mosaic, compressed via the reference block coder (RBC), and accessed and delivered over the Internet through the virtual media (Vmedia) access protocol. Capturing the environment through the concentric mosaic is easy. We mount a camera at the end of a level beam, and shoot images as the beam rotates. The huge dataset of the concentric mosaic is then compressed through the RBC, which is specifically designed for both high compression efficiency and just-in-time (JIT) rendering. Through the JIT rendering function, only a portion of the RBC bitstream is accessed, decoded and rendered for each virtual view. A multimedia communication protocol -- the Vmedia protocol, is then proposed to deliver the compressed concentric mosaic data over the Internet. Only the bitstream segments corresponding to the current view are streamed over the Internet. Moreover, the delivered bitstream segments are managed by a local Vmedia cache so that frequently used bitstream segments need not be streamed over the Internet repeatedly, and the Vmedia is able to handle a RBC bitstream larger than its memory capacity. A Vmedia concentric mosaic interactive browser is developed where the user can freely wander in a realistic environment, e.g., rotate around, walk forward/backward and sidestep, even under a tight bandwidth of 33.6 kbps.
Concentric mosaics have the ability to quickly capture a complete 3D view of a realistic environment and to enable a user to wander freely in the environment. However, the data amount of the concentric mosaics is huge. In this paper, we propose an algorithm to compress the concentric mosaic image array through motion compensation and residue coding, which we called reference block codec. A two-level index table is embedded in the compressed bitstream for random access. During the rendering, the entire compressed concentric mosaic scene is not fully expanded at any time. Instead, only the contents necessary to render the current view are decoded in real time. We denote such rendering scheme as just-in-time rendering. Four decoder caches are implemented to speed up the rendering.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.