Abstract: Physically Based Rendering: From Theory to Implementation, Third Edition, describes both the mathematical theory behind a modern photorealistic rendering system and its practical implementation. Through a method known as 'literate programming', the authors combine human-readable documentation and source code into a single reference that is specifically designed to aid comprehension. The result is a stunning achievement in graphics education. Through the ideas and software in this book, users will learn to design and employ a fully-featured rendering system for creating stunning imagery. This completely updated and revised edition includes new coverage on ray-tracing hair and curves primitives, numerical precision issues with ray tracing, LBVHs, realistic camera models, the measurement equation, and much more. It is a must-have, full color resource on physically-based rendering. Presents up-to-date revisions of the seminal reference on rendering, including new sections on bidirectional path tracing, numerical robustness issues in ray tracing, realistic camera models, and subsurface scattering Provides the source code fora complete rendering systemallowing readers to get up and running fast Includes a unique indexing feature, literate programming, that lists the locations of each function, variable, and method on the page where they are first describedServes as an essential resource on physically-based rendering

Abstract: A multidimensional image navigation and display software was designed for display and interpretation of large sets of multidimensional and multimodality images such as combined PET-CT studies. The software is developed in Objective-C on a Macintosh platform under the MacOS X operating system using the GNUstep development environment. It also benefits from the extremely fast and optimized 3D graphic capabilities of the OpenGL graphic standard widely used for computer games optimized for taking advantage of any hardware graphic accelerator boards available. In the design of the software special attention was given to adapt the user interface to the specific and complex tasks of navigating through large sets of image data. An interactive jog-wheel device widely used in the video and movie industry was implemented to allow users to navigate in the different dimensions of an image set much faster than with a traditional mouse or on-screen cursors and sliders. The program can easily be adapted for very specific tasks that require a limited number of functions, by adding and removing tools from the program’s toolbar and avoiding an overwhelming number of unnecessary tools and functions. The processing and image rendering tools of the software are based on the open-source libraries ITK and VTK. This ensures that all new developments in image processing that could emerge from other academic institutions using these libraries can be directly ported to the OsiriX program. OsiriX is provided free of charge under the GNU open-source licensing agreement at http://homepage.mac.com/rossetantoine/osirix.

Abstract: The ability to interactively control viewpoint while watching a video is an exciting application of image-based rendering. The goal of our work is to render dynamic scenes with interactive viewpoint control using a relatively small number of video cameras. In this paper, we show how high-quality video-based rendering of dynamic scenes can be accomplished using multiple synchronized video streams combined with novel image-based modeling and rendering algorithms. Once these video streams have been processed, we can synthesize any intermediate view between cameras at any time, with the potential for space-time manipulation.In our approach, we first use a novel color segmentation-based stereo algorithm to generate high-quality photoconsistent correspondences across all camera views. Mattes for areas near depth discontinuities are then automatically extracted to reduce artifacts during view synthesis. Finally, a novel temporal two-layer compressed representation that handles matting is developed for rendering at interactive rates.

Abstract: This paper presents details of a system that allows for an evolutionary introduction of depth perception into the existing 2D digital TV framework. The work is part of the European Information Society Technologies (IST) project “Advanced Three-Dimensional Television System Technologies” (ATTEST), an activity, where industries, research centers and universities have joined forces to design a backwards-compatible, flexible and modular broadcast 3D-TV system. At the very heart of the described new concept is the generation and distribution of a novel data representation format, which consists of monoscopic color video and associated perpixel depth information. From these data, one or more “virtual” views of a real-world scene can be synthesized in real-time at the receiver side (i. e. a 3D-TV set-top box) by means of so-called depth-image-based rendering (DIBR) techniques. This publication will provide: (1) a detailed description of the fundamentals of this new approach on 3D-TV; (2) a comparison with the classical approach of “stereoscopic” video; (3) a short introduction to DIBR techniques in general; (4) the development of a specific DIBR algorithm that can be used for the efficient generation of high-quality “virtual” stereoscopic views; (5) a number of implementation details that are specific to the current state of the development; (6) research on the backwards-compatible compression and transmission of 3D imagery using state-of-the-art MPEG (Moving Pictures Expert Group) tools.

Abstract: Rendering throughput has reached a level that enables a novel approach to level-of-detail (LOD) control in terrain rendering. We introduce the geometry clipmap, which caches the terrain in a set of nested regular grids centered about the viewer. The grids are stored as vertex buffers in fast video memory, and are incrementally refilled as the viewpoint moves. This simple framework provides visual continuity, uniform frame rate, complexity throttling, and graceful degradation. Moreover it allows two new exciting real-time functionalities: decompression and synthesis. Our main dataset is a 40GB height map of the United States. A compressed image pyramid reduces the size by a remarkable factor of 100, so that it fits entirely in memory. This compressed data also contributes normal maps for shading. As the viewer approaches the surface, we synthesize grid levels finer than the stored terrain using fractal noise displacement. Decompression, synthesis, and normal-map computations are incremental, thereby allowing interactive flight at 60 frames/sec.

Abstract: Haptic rendering allows users to "feel" virtual objects in a simulated environment. We survey current haptic systems and discuss some basic haptic-rendering algorithms. In the past decade we've seen an enormous increase in interest in the science of haptics. Haptics broadly refers to touch interactions (physical contact) that occur for the purpose of perception or manipulation of objects. These interactions can be between a human hand and a real object; a robot end-effector and a real object; a human hand and a simulated object (via haptic interface devices); or a variety of combinations of human and machine interactions with real, remote, or virtual objects. Rendering refers to the process by which desired sensory stimuli are imposed on the user to convey information about a virtual haptic object.

Abstract: We consider a dynamic social network model in which agents play repeated games in pairings determined by a stochastically evolving social network. Individual agents begin to interact at random, with the interactions modeled as games. The game payoffs determine which interactions are reinforced, and the network structure emerges as a consequence of the dynamics of the agents' learning behavior. We study this in a variety of game-theoretic conditions and show that the behavior is complex and sometimes dissimilar to behavior in the absence of structural dynamics. We argue that modeling network structure as dynamic increases realism without rendering the problem of analysis intractable.

Abstract: In a recent paper Dinur and Nissim considered a statistical database in which a trusted database administrator monitors queries and introduces noise to the responses with the goal of maintaining data privacy [5]. Under a rigorous definition of breach of privacy, Dinur and Nissim proved that unless the total number of queries is sub-linear in the size of the database, a substantial amount of noise is required to avoid a breach, rendering the database almost useless.

Abstract: The present invention is directed to a method and system for automatic calibration of an acoustic system. The acoustic system may include a source A/V device, calibration computing device, and multiple rendering devices. The calibration system may include a calibration component attached to each rendering device and a source calibration module. The calibration component on each rendering device includes a microphone. The source calibration module includes distance and optional angle calculation tools for automatically determining a distance between the rendering device and a specified reference point upon return of the test signal from the calibration component.

Abstract: A light field consists of images of a scene taken from different viewpoints. Light fields are used in computer graphics for image-based rendering and synthetic aperture photography, and in vision for recovering shape. In this paper, we describe a simple procedure to calibrate camera arrays used to capture light fields using a plane + parallax framework. Specifically, for the case when the cameras lie on a plane, we show (i) how to estimate camera positions up to an affine ambiguity, and (ii) how to reproject light field images onto a family of planes using only knowledge of planar parallax for one point in the scene. While planar parallax does not completely describe the geometry of the light field, it is adequate for the first two applications which, it turns out, do not depend on having a metric calibration of the light field. Experiments on acquired light fields indicate that our method yields better results than full metric calibration.

Abstract: As the Red Book is known to be the gold standard for OpenGL, the Orange Book is considered to be the gold standard for the OpenGL Shading Language. With Randis extensive knowledge of OpenGL and GLSL, you can be assured you will be learning from a graphics industry veteran. Within the pages of the second edition you can find topics from beginning shader development to advanced topics such as the spherical harmonic lighting model and more. David Tommeraasen, CEO/Programmer, Plasma Software This will be the definitive guide for OpenGL shaders; no other book goes into this detail. Rost has done an excellent job at setting the stage for shader development, what the purpose is, how to do it, and how it all fits together. The book includes great examples and details, as well as good additional coverage of 2.0 changes! Jeffery Galinovsky, Director of Emerging Market, Platform Development, Intel Corporation The coverage in this new edition of the book is pitched just right to help many new shader-writers get started, but with enough deep information for the old hands. Marc Olano, Assistant Professor, University of Maryland This is a really great book on GLSLwell written and organized, very accessible, and with good real-world examples and sample code. The topics flow naturally and easily, explanatory code fragments are inserted in very logical places to illustrate concepts, and, all in all, this book makes an excellent tutorial as well as a reference. John Carey, Chief Technology Officer, C.O.R.E. Feature Animation OpenGL Shading Languageprovides a timely, thorough, and entertaining introduction to the only OpenGL ARB-approved high-level shading language in existence. Whether an expert or a novice, there are gems to be discovered throughout the book, and the reference pages will be your constant companion as you dig into the depths of the shading APIs. From algorithms to APIs, this book has you covered. Bob Kuehne, CEO, Blue Newt Software Computer graphics and rendering technologies just took a giant leap forward with hardware vendors rapidly adopting the new OpenGL Shading Language. This book presents a detailed treatment of these exciting technologies in a way that is extremely helpful for visualization and game developers. Andy McGovern, Founder, Virtual Geographics, Inc. The OpenGL Shading Language is at the epicenter of the programmable graphics revolution, and Randi Rost has been at the center of the development of this significant new industry standard. If you need the inside track on how to use the OpenGL Shading Language to unleash new visual effects and unlock the supercomputer hiding inside the new generation of graphics hardware, then this is the book for you. Neil Trevett, Senior Vice President, Market Development, 3Dlabs OpenGL Shading Language, Third Edition, extensively updated for OpenGL 3.1, is the experienced application programmers guide to writing shaders. Part reference, part tutorial, this book thoroughly explains the shift from fixed-functionality graphics hardware to the new era of programmable graphics hardware and the additions to the OpenGL API that support this programmability. With OpenGL and shaders written in the OpenGL Shading Language, applications can perform better, achieving stunning graphics effects by using the capabilities of both the visual processing unit and the central processing unit. In this book, you will find a detailed introduction to the OpenGL Shading Language (GLSL) and the new OpenGL function calls that support it. The text begins by describing the syntax and semantics of this high-level programming language. Once this foundation has been established, the book explores the creation and manipulation of shaders using new OpenGL function calls. OpenGL Shading Language, Third Edition, includes updated descriptions for the language and all the GLSL entry points added though OpenGL 3.1, as well as updated chapters that discuss transformations, lighting, shadows, and surface characteristics. The third edition also features shaders that have been updated to OpenGL Shading Language Version 1.40 and their underlying algorithms, including Traditional OpenGL fixed functionality Stored textures and procedural textures Image-based lighting Lighting with spherical harmonics Ambient occlusion and shadow mapping Volume shadows using deferred lighting Wards BRDF model The color plate section illustrates the power and sophistication of the OpenGL Shading Language. The API Function Reference at the end of the book is an excellent guide to the API entry points that support the OpenGL Shading Language.

Abstract: The MLS surface [Levin 2003], used for modeling and rendering with point clouds, was originally defined algorithmically as the output of a particular meshless construction. We give a new explicit definition in terms of the critical points of an energy function on lines determined by a vector field. This definition reveals connections to research in computer vision and computational topology.Variants of the MLS surface can be created by varying the vector field and the energy function. As an example, we define a similar surface determined by a cloud of surfels (points equipped with normals), rather than points.We also observe that some procedures described in the literature to take points in space onto the MLS surface fail to do so, and we describe a simple iterative procedure which does.

Abstract: A system and method in a client-server computing environment is provided for generating and rendering a photorealistic three-dimensional (3D) perspective view of a 3D object selectively positioned within a 3D scene. A client is configured for communicably accessing a server, and includes a client application configured for scene editing and rendering, using a graphical user interface (GUI). The 3D scene may be selectively displayed in a plurality of views, and a 3D object may be retrieved from the server and imported into the 3D scene to generate a composite. The 3D object may also be manipulated within the composite for placement and orientation. A 3D image of the composite may then be rendered at the client and selectively reconfigured in real time. Luminosity characteristics may be applied to the 3D image; followed by rendering a photorealistic 3D view of the composite image.

Abstract: Systems and methods for balancing a load among multiple graphics processors that render different portions of a frame A display area is partitioned into portions for each of two (or more) graphics processors The graphics processors render their respective portions of a frame and return feedback data indicating completion of the rendering Based on the feedback data, an imbalance can be detected between respective loads of two of the graphics processors In the event that an imbalance exists, the display area is re-partitioned to increase a size of the portion assigned to the less heavily loaded processor and to decrease a size of the portion assigned to the more heavily loaded processor

Abstract: Many cultural heritage applications require 3D reconstruction of real-world objects and scenes. Over the past few years, it has become increasingly common to use 3D digitization and modeling for this purpose. This is mainly due to advances in laser-scanning techniques, 3D modeling software, image-based modeling techniques, computer power, and virtual reality. Our approach integrates several technologies based on our experience over more than a decade of trying to accurately and completely model large-scale heritage monuments and sites. Using both interactive and automatic techniques, we can model a highly detailed structure or site at various levels of detail. We use image-based modeling for basic shape and structural elements, and laser scanning for fine details and sculpted surfaces. To present the site in its proper context, we use image-based rendering for landscapes and surroundings. To apply this approach, we created hundreds of models from sites all over the world for documentation, walk-through movies, and interactive visualization. The results were compelling and encouraging.

Abstract: A media server (302) in a Universal Plug and Play network (300) includes a resource sharing service (322) to govern the distribution of resource information regarding resources to rendering devices (304-312). In one case, the resource sharing service (322) consults a creiterion to determine whether an identified network device is authorized to receive resource information. In another case, the resource sharing service (322) consults another criterion to determine whether a specified individual associated with the media server (302) must consent to the transfer of the resource information in order for the transfer to occur. The resource information may include resource metadata that describes high level information regarding resources, as well as resource content. The media server (302) includes various user interface presentations that allow the media server user to specify shared resources and distribution criteria.

Abstract: The present invention provides an automated interactive system that enables an author to build applications that handle complex consumer-merchant interactions. The author designs content components and application rules that are interpreted by the system dynamically, at runtime, to generate and deliver to users personalized HTML web pages, including client-side objects that track user behavior and enhance users' interaction with the application. Such rules dynamically adapt the author's goals to the appropriate users at the appropriate time, thereby simulating the dialogue between users and human salespersons/customer service personnel in the context of an automated interactive system.

Abstract: Creating an account via a mobile device. A request to create an account is received from a user via a mobile device. The request is received from the mobile device according to a wireless communications protocol. In response to the received request, a confirmation message is delivered to the mobile device for rendering to the user as a challenge. A response to the rendered challenge is received from the user via the mobile device. It is determined if the received response to the rendered challenge corresponds to the delivered confirmation message. And the account is created for the user if the received response to the rendered challenge is determined to correspond to the delivered confirmation message. Other aspects of the invention are directed to computer-readable media for use with account creation via a mobile device.

Abstract: We present a simple but powerful algorithm for optimizing the usage of hardware occlusion queries in arbitrary complex scenes. Our method minimizes the number of issued queries and reduces the delays due to the latency of query results. We reuse the results of occlusion queries from the last frame in order to initiate and schedule the queries in the next frame. This is done by processing nodes of a spatial hierarchy in a front-to-back order and interleaving occlusion queries with rendering of certain previously visible nodes. The proposed scheduling of the queries makes use of spatial and temporal coherence of visibility. Despite its simplicity, the algorithm achieves good culling efciency for scenes of various types. The implementation of the algorithm is straightforward and it can be easily integrated in existing real-time rendering packages based on common hierarchical data structures.

Abstract: This paper introduces a tensor framework for image-based rendering. In particular, we develop an algorithm called TensorTextures that learns a parsimonious model of the bidirectional texture function (BTF) from observational data. Given an ensemble of images of a textured surface, our nonlinear, generative model explicitly represents the multifactor interaction implicit in the detailed appearance of the surface under varying photometric angles, including local (per-texel) reflectance, complex mesostructural self-occlusion, interreflection and self-shadowing, and other BTF-relevant phenomena. Mathematically, TensorTextures is based on multilinear algebra, the algebra of higher-order tensors, hence its name. It is computed through a decomposition known as the N-mode SVD, an extension to tensors of the conventional matrix singular value decomposition (SVD). We demonstrate the application of TensorTextures to the image-based rendering of natural and synthetic textured surfaces under continuously varying viewpoint and illumination conditions.

Abstract: High-quality virtual audio scene rendering is required for emerging virtual and augmented reality applications, perceptual user interfaces, and sonification of data. We describe algorithms for creation of virtual auditory spaces by rendering cues that arise from anatomical scattering, environmental scattering, and dynamical effects. We use a novel way of personalizing the head related transfer functions (HRTFs) from a database, based on anatomical measurements. Details of algorithms for HRTF interpolation, room impulse response creation, HRTF selection from a database, and audio scene presentation are presented. Our system runs in real time on an office PC without specialized DSP hardware.

Abstract: This paper presents a self-reconfigurable camera array system that captures video sequences from an array of mobile cameras, renders novel views on the fly and reconfigures the camera positions to achieve better rendering quality. The system is composed of 48 cameras mounted on mobile platforms. The contribution of this paper is twofold. First, we propose an efficient algorithm that is capable of rendering high-quality novel views from the captured images. The algorithm reconstructs a view-dependent multi-resolution 2D mesh model of the scene geometry on the fly and uses it for rendering. The algorithm combines region of interest (ROI) identification, JPEG image decompression, lens distortion correction, scene geometry reconstruction and novel view synthesis seamlessly on a single Intel Xeon 2.4 GHz processor, which is capable of generating novel views at 4–10 frames per second (fps). Second, we present a view-dependent adaptive capturing scheme that moves the cameras in order to show even better rendering results. Such camera reconfiguration naturally leads to a nonuniform arrangement of the cameras on the camera plane, which is both view-dependent and scene-dependent.

Abstract: Reconstructing and rendering trees is a challenging problem due to the geometric complexity involved, and the inherent difficulties of capture. In this paper we propose a volumetric approach to capture and render trees with relatively sparse foliage. Photographs of such trees typically have single pixels containing the blended projection of numerous leaves/branches and background. We show how we estimate opacity values on a recursive grid, based on alphamattes extracted from a small number of calibrated photographs of a tree. This data structure is then used to render billboards attached to the centers of the grid cells. Each billboard is assigned a set of view-dependent textures corresponding to each input view. These textures are generated by approximating coverage masks based on opacity and depth from the camera. Rendering is performed using a view-dependent texturing algorithm. The resulting volumetric tree structure has low polygon count, permitting interactive rendering of realistic 3D trees. We illustrate the implementation of our system on several different real trees, and show that we can insert the resulting model in virtual scenes.

Abstract: A method is provided for customizing scene content, according to a user or a cluster of users, for a given stereoscopic display, including obtaining customization information about the user; obtaining a scene disparity map for a pair of given stereo images and/or a three-dimensional (3D) computer graphic model; and determining an aim disparity range for the user. The method of the present invention also generates a customized disparity map and/or rendering conditions for a three-dimensional (3D) computer graphic model correlating with the user's fusing capability of the given stereoscopic display; and renders or re-renders the stereo images for subsequent display.

Abstract: Ambiguities with respect to a user topic interest may be resolved so that useful topic-relevant ads can be presented. Such ambiguities may be resolved by monitoring user behavior, determining a user topic interest (e.g., from a plurality of different candidate topics) based on the monitored behavior, and serving ads relevant to the determined user topic interest.

Abstract: Despite the notable progress in physically-based rendering, there is still a long way to go before we can automatically generate predictable images of biological materials. In this paper, we address an open problem in this area, namely the spectral simulation of light interaction with human skin. We propose a novel biophysicallybased model that accounts for all components of light propagation in skin tissues, namely surface reflectance, subsurface reflectance and transmittance, and the biological mechanisms of light absorption by pigments in these tissues. The model is controlled by biologically meaningful parameters, and its formulation, based on standard Monte Carlo techniques, enables its straightforward incorporation into realistic image synthesis frameworks. Besides its biophysically-based nature, the key difference between the proposed model and the existing skin models is its comprehensiveness, i.e., it computes both spectral (reflectance and transmittance) and scattering (bidirectional surface-scattering distribution function) quantities for skin specimens. In order to assess the predictability of our simulations, we evaluate their accuracy by comparing results from the model with actual skin measured data. We also present computer generated images to illustrate the flexibility of the proposed model with respect to variations in the biological input data, and its applicability not only in the predictive image synthesis of different skin tones, but also in the spectral simulation of medical conditions.

Abstract: Valuable 3D graphical models, such as high-resolution digital scans of cultural heritage objects, may require protection to prevent piracy or misuse, while still allowing for interactive display and manipulation by a widespread audience. We have investigated techniques for protecting 3D graphics content, and we have developed a remote rendering system suitable for sharing archives of 3D models while protecting the 3D geometry from unauthorized extraction. The system consists of a 3D viewer client that includes low-resolution versions of the 3D models, and a rendering server that renders and returns images of high-resolution models according to client requests. The server implements a number of defenses to guard against 3D reconstruction attacks, such as monitoring and limiting request streams, and slightly perturbing and distorting the rendered images. We consider several possible types of reconstruction attacks on such a rendering server, and we examine how these attacks can be defended against without excessively compromising the interactive experience for non-malicious users.