Abstract: Image-based rendering (IBR) is unique in that it requires computer graphics, computer vision, and image processing to join forces to solve a common goal, namely photorealistic rendering through the use of images. IBR as an area of research has been around for about ten years, and substantial progress has been achieved in effiectively capturing, representing, and rendering scenes. In this article, we survey the techniques used in IBR. Our survey shows that representations and rendering techniques can differ radically, depending on design decisions related to ease of capture, use of geometry, accuracy of geometry (if used), number and distribution of source images, degrees of freedom for virtual navigation, and expected scene complexity.

Abstract: In this paper we present a novel method for plausible real-time rendering of indirect illumination effects for diffuse and non-diffuse surfaces. The scene geometry causing indirect illumination is captured by an extended shadow map, as proposed in previous work, and secondary light sources are distributed on directly lit surfaces. One novelty is the rendering of these secondary lights' contribution by splatting in a deferred shading process, which decouples rendering time from scene complexity. An importance sampling strategy, implemented entirely on the GPU, allows efficient selection of secondary light sources. Adapting the light's splat shape to surface glossiness also allows efficient rendering of caustics. Unlike previous approaches the approximated indirect lighting does barely exhibit coarse artifacts - even under unfavorable viewing and lighting conditions. We describe an implementation on contemporary graphics hardware, show a comparison to previous approaches, and present adaptation to and results in game-typical applications.

Abstract: A system and method is presented that leverages independent innovation in entertainment content and graphics hardware. In this system and method, the current image generation run-time application is replaced with a new framework defining the connectivity, features, and behavior necessary to implement a graphics system. All this takes place in the context of a software platform utilizing a late-integration mechanism that dynamically integrates the various real-time components in a run-time application. Ultimately displacing hardware as the central focus of development efforts, this software platform functionally is the graphics application, at least as viewed by the simulation host computer, database developers, and those responsible for visual system procurement and maintenance. An innovative software architecture, the Graphical Application Platform (GAP) is presented. The GAP builds on image generator, workstation, and scene graph success by extending the concepts of platform and framework into the real-time graphics domain—bridging the gap between image generation concerns and contemporary hardware and software realities by decoupling content, hardware and applications. This new approach also provides technology to address emerging concerns related to the selection and acquisition processes in the context of new low-cost, high-performance graphics hardware.

Abstract: The world of 3D graphics, until recently restricted to high-end workstations and game consoles, is rapidly expanding into the domain of mobile platforms such as cellular phones and PDAs. Even as the mobile chip market is poised to exceed production of 500 million chips per year, incorporation of 3D graphics in handhelds poses several serious challenges to the hardware designer. Compared with other platforms, graphics on handhelds have to contend with limited energy supplies and lower computing horsepower. Nevertheless, images must still be rendered at high quality since handheld screens are typically held closer to the observer’s eye, making imperfections and approximations very noticeable. In this paper, we provide an in-depth quantitative analysis of the power consumption of mobile 3D graphics pipelines. We analyze the effects of various 3D graphics factors such as resolution, frame rate, level of detail, lighting and texture maps on power consumption. We demonstrate that significant imbalance exists across the workloads of different graphics pipeline stages. In addition, we illustrate how this imbalance may vary dynamically, depending on the characteristics of the graphics application. Based on this observation, we identify and compare the benefits of candidate Dynamic Voltage and Frequency Scaling (DVFS) schemes for mobile 3D graphics pipelines. In our experiments we observe that DVFS for mobile 3D graphics reduces energy by as much as 50%.

Abstract: We introduce a parallel, distributed memory algorithm for volume rendering massive data sets. The algorithm's scalability has been demonstrated up to 400 processors, rendering one hundred million unstructured elements in under one second. The heart of the algorithm is a hybrid approach that parallelizes over both the elements of the input data and over the pixels of the output image. At each stage of the algorithm, there are strong limits on how much work each processor performs, ensuring good parallel efficiency. The algorithm is sample-based. We present two techniques for calculating the sample points: a 3D rasterization technique and a kernel-based technique, which trade off between speed and generality. Finally, the algorithm is very flexible. It can be deployed in general purpose visualization tools and can also support diverse mesh types, ranging from structured grids to curvilinear and unstructured meshes to point clouds.

Abstract: Since graphics cards have become programmable the recent years, numerous computationally intensive algorithms have been implemented on the now called General Purpose Graphics Processing Units (GPGPUs). While the results show that GPGPUs regularly outperform CPU based implementations, the question arose how optical flow algorithms can be ported to graphics hardware. To answer the question, the optimal algorithm structure to maximize the performance gained by using graphics cards has to be found. In this paper we compare the performance of two algorithms that are highly different in structure, implemented on both CPU and graphics hardware. Analyzing the results of the CPU and GPGPU implementation, we explore the mapping of the algorithms to the graphics hardware and thereof extract information about a preferred structure of optical flow algorithms for GPGPU based implementation.

Abstract: We describe a terrain rendering algorithm for spherical terrains based on clipmaps. It leverages the high geometry throughput of current GPU to render large static triangle sets. The vertices are displaced by a height map texture. Our main contribution is mapping of texture coordinates to calculate the height map sample position based on the static vertex offset and the variable view position.

Abstract: This book presents efficient visualization techniques, a prerequisite for the interactive exploration of complex data sets. High performance is demonstrated as a process of devising algorithms for the fast graphics processing units (GPUs) of modern graphics hardware. Coverage includes parallelization on cluster computers with several GPUs, adaptive rendering methods, and non-photorealistic rendering techniques for visualization.

Abstract: Rendering packages are used by visual psychophysicists to produce complex stimuli for their experiments, tacitly assuming that the simulation results accurately reflect the light-surface interactions of a real scene. RADIANCE is a physically based, freely available, and commonly used rendering software. We validated the calculation accuracy of this package by comparing simulation results with measurements from real scenes. RADIANCE recovers color gradients well but the results are shifted in color space. Currently, there is no better simulation alternative for achieving physical accuracy than by combining a spectral rendering method with RADIANCE.

Abstract: Graphics resources are virtualized through an interface between graphics hardware and graphics clients. The interface allocates the graphics resources across multiple graphics clients, processes commands for access to the graphics resources from the graphics clients, and resolves conflicts for the graphics resources among the clients.

Abstract: The availability of more powerful mobile devices, sometimes equipped with graphics accelerators, is making it easier to experiment with mobile 3D graphics. In this paper, we exploit the main emerging standard in 3D rendering on mobile devices (OpenGL ES) to build a mobile player (called MobiX3D) for X3D and H-Anim content. The rendering engine of the MobiX3D player supports classic lighting and shading algorithms. We discuss the performance of the player and we apply it to sign language visualization.

Abstract: The most efficient general occlusion culling techniques are based on hardware accelerated occlusion queries. Although in many cases these techniques can considerably improve performance, they may still reduce efficiency compared to simple view frustum culling, especially in the case of low depth complexity. This prevented the broad use of occlusion culling in most commercial applications. In this paper we present a new conservative method to solve this problem, where the main idea is to use a statistical model describing the occlusion probability for each occlusion query in order to reduce the number of wasted queries which are the reason for the reduction in rendering speed. We also describe an abstract parameterized model for the graphics hardware performance. The parameters are easily measurable at startup and thus the model can be adapted to the graphics hardware in use. Combining this model with the estimated occlusion probability our method is able to achieve a near optimal scheduling of the occlusion queries. The implementation of the algorithm is straightforward and it can be easily integrated in existing real-time rendering packages based on common hierarchical data structures.

Abstract: Computer graphics textbooks teach that sampling by deterministic patterns or even lattices causes aliasing, which only can be avoided by random, i.e. independent sampling. They recommend random samples with blue noise characteristic, which however are highly correlated due to their maximized minimum mutual distance. On the other hand the rendering software mental ray, which is used to generate the majority of visual effects in movies, entirely is based on parametric integration by quasi-Monte Carlo methods and consequently is strictly deterministic. For its superior quality the software even received a Technical Achievement Award (Oscar) by the American Academy of Motion Picture Arts and Sciences in 2003. Along the milestones of more than ten years of development of quasi-Monte Carlo methods in computer graphics, we point out that the two previous statements are not contradictory.

Abstract: Parallel volume rendering is one of the most efficient techniques to achieve real time visualization of large datasets by distributing the data and the rendering process over a cluster of machines. However, when using level of detail techniques or when zooming on parts of the datasets, load unbalance becomes a challenging issue that has not been widely studied in the context of hardware-based rendering. In this paper, we address this issue and show how to achieve good load balancing for parallel level of detail volume rendering. We do so by dynamically distributing the data among the rendering nodes according to the load of the previous frame. We illustrate the efficiency of our technique on large datasets.

Abstract: Direct Volume Rendering (DVR) is a technique for creating images directly from a representation of a function defined over a three-dimensional domain. The technique has many application fields, such as scientific visualization and medical imaging. A striking property of the data sets produced within these fields is their ever increasing size and complexity. Despite the advancements of computing resources these data sets seem to grow at even faster rates causing severe bottlenecks in terms of data transfer bandwidths, memory capacity and processing requirements in the rendering pipeline. This thesis focuses on efficient methods for DVR of large data sets. At the core of the work lies a level-of-detail scheme that reduces the amount of data to process and handle, while optimizing the level-of-detail selection so that high visual quality is maintained. A set of techniques for domain knowledge encoding which significantly improves assessment and prediction of visual significance for blocks in a volume are introduced. A complete pipeline for DVR is presented that uses the data reduction achieved by the level-of-detail selection to minimize the data requirements in all stages. This leads to reduction of disk I/O as well as host and graphics memory. The data reduction is also exploited to improve the rendering performance in graphics hardware, employing adaptive sampling both within the volume and within the rendered image. The developed techniques have been applied in particular to medical visualization of large data sets on commodity desktop computers using consumer graphics processors. The specific application of virtual autopsies has received much interest, and several developed data classification schemes and rendering techniques have been motivated by this application. The results are, however, general and applicable in many fields and significant performance and quality improvements over previous techniques are shown.

Abstract: Rendering computer animation frames is a very time consuming job. Using parallel computing on clusters and so-called render farms is a common solution to this problem. In this paper we describe how Grid computing can be used for computer animation rendering. We propose a framework for Grid rendering services, describe its implementation, and present the results and statistics. A loseless 3D compression algorithm was also devised to solve the existing problem of transferring gigabytes of scene representation files (Renderman (.rib) and mental images (.mi)). This compression algorithm has been filed for patent in Singapore.

Abstract: A high performance graphics processing and display system architecture realized on a monolithic silicon chip, supporting a cluster of multiple cores of graphic processing units (GPUs) that cooperate to provide a powerful and highly scalable visualization solution supporting photo-realistic graphics capabilities for diverse applications. The present invention eliminates rendering bottlenecks along the graphics pipeline by dynamically managing various parallel rendering techniques and enabling adaptive handling of diverse graphics applications.

Abstract: A multi-GPU rendering system according to a preferred embodiment of the present invention includes a CPU, a chipset, the first GPU (graphics processing unit), the first graphics memory for the first GPU, a second GPU, and the second graphics memory for the second GPU. The chipset is electrically connected to the CPU, the first GPU and the second GPU. Graphics content is divided into two parts for the two GPUs to process separately. The two parts of the graphics content may be the same or different in sizes. Two processed graphics results are combined in one of these two graphics memories to form complete image stream and then it is outputted to a display by the GPU.

Abstract: Resolution independent rendering is important for many applications such as text rendering and rendering vector images. This area has received quite some interest in recent years due to the growing popularity of Flash and SVG-based applications. This sketch presents a new method for resolution independent rendering of vector images suitable for programmable graphics hardware. We have enhanced a previous method [Loop and Blinn 2005] by using a stencil buffer and transparency multisampling [ATI 2005; NVI 2004], so that our method has the following advantages:

Abstract: The fundamental drawback of current stereo and multi-view visualization is the necessity to perform multi pass rendering (one pass for each view) and subsequent image composition + masking for generating multiple stereo views. Thus the rendering time increases in general linearly with the number of views.In this paper we introduce a new method for multi-view splatting based on deferred blending. Our method exploits the programma-bility of modern graphic processing units (GPUs) for rendering multiple stereo views in a single rendering pass. The views are calculated directly on the GPU including sub-pixel wavelength selective views. We describe our algorithm precisely and provide details about its implementation. Experimental results demonstrate the performance advantage of our multi-view point splatting algorithm compared to the standard multi-pass approach.

Abstract: — In this paper, methods and instruments for measurement and evaluation of reflection characteristics are reviewed as needed for research and development of electronic displays and for material and surface modeling with ray-tracing and rendering software packages. Contrast under ambient illumination and recognizability under daylight illumination are prime development targets in the electronic-display field, while computation and synthesis of realistic scenes and objects are pushing the need for physical data in computer graphics applications. Three categories of instruments are available for detailed reflection analysis. They are based on (1) gonioscopic (mechanical) and conoscopic (optical) directional scanning, (2) imaging approaches, and (3) on arrangements with variable source or receiver aperture. The capabilities, advantages, and limitations of these methods are introduced and discussed in order to facilitate appropriate selection of methods and instruments. For illustration purposes we present typical results obtained from commercial electronic display screens. A basis for continued widespread implementation and standardization of reflection metrology as required for objective rating and comparison of electronic-display screen performance under ambient illumination is provided.

Abstract: A method, apparatus, and article of manufacture provide the ability to conduct global illumination. A three-dimensional (3D) model of a scene is obtained in a computer graphics application. A section of the scene is identified as a region of interest. A photon tree is then obtained that consists of a set of buffers that represents the region of interest, with every pixel in the region of interest necessary for every view being represented in at least one buffer in the set of buffers. The set of buffers are concatenated into a single large buffer. One or more full screen draw operations is performed over the single large buffer. The draw operation performs a lighting and optional shadowing operation on every pixel represented in the set of buffers. Any view of the region of interest is then displayed based on the lighting information thus incorporated into the photon tree.

Abstract: (a) (b) Figure 1. Realistic city scene rendered using parallax occlusion mapping applied to the cobblestone sidewalk in (a) and using the normal mapping technique in (b).

Abstract: Various embodiments are disclosed relating to providing a pixel history for a graphics application. During rendering of a visual representation, such as a computer game or visual simulation, a developer or other user may observe a rendering error, e.g., with respect to a rendered pixel, or may wish to optimize or understand an operation of the visual representation. The developer may select the pixel and be provided with a browsable pixel history window that shows a temporal, sequential order of events associated with the rendering of the selected pixel. The events may include calls from the graphics application to an associated graphics interface, and information about the calls may include asset data associated with the calls as well as primitives associated with the calls.

Abstract: The photo-realistic reproduction of natural terrains is a classical challenge in computer graphics and the interactive display of non-trivial landscapes is only possible with recent graphics hardware. The reasons for this are the vast amount of data due to geometric detail of the terrain, vegetation and further objects, but also the inherent complexity of natural phenomena that are necessary to achieve convincing results. Realistic terrain rendering also has to consider the complex lighting conditions due to atmospheric scattering and further aspects such as the rendering of waterbodies and cloudscapes. One focus of this dissertation are procedural models for terrain elevation and texturing. They can either be used to create completely artificial, realistic landscapes, mimic real terrains by guiding the models with real-world data or augment acquired data with additional procedural detail. By this, we can reproduce natural scenes with the advantage of a compact procedural description. Another emphasis is placed on the interactive rendering of such terrains including specifically tailored level-of-detail methods and lighting computations for terrains and rendering techniques for complex plant models. We introduce a set of novel techniques and algorithms that address the aforementioned problems and achieve results in real-time with photo-realistic image quality using programmable graphics hardware. In particular, we propose new algorithms for a data-guided height field creation, realistic terrain surface texture generation, a novel level-of-detail method for terrain rendering and hardware friendly, efficient point-based rendering and splatting techniques. We also provide a comprehensive presentation of underlying theories and related work to put our work in perspective of the research area.

Abstract: An apparatus for conducting simulations and presenting the results in real-time or substantially in real-time to a user comprises one or more CPUs operating in parallel with one or more GPUs. CPU software controls user interface functions and GPU software effects the simulations. Users can interact with the simulation as the simulation is occurring. A non-batch graphics image may be rendered and displays by calculating a first set of parameters in the CPU; initializing, in the CPU, a model for graphics display; generating a texture for display in the CPU; passing the generated texture from the CPU to the GPU via a bus; providing compiled graphics code to the GPU; calculating a timestep in the GPU; updating the passed, generated texture in the GPU; passing the updated texture to a a memory comprising a framebuffer; reading desired contents from the framebuffer by the CPU; and displaying a new property.

Abstract: With the emergence of 3D graphics/arts assets commerce on the Internet, to protect their intellectual property and to restrict their usage have become a new design challenge. This paper presents a novel protection model for commercial graphics data by integrating digital rights management into the graphics processing unit and creating a digital rights enabled graphics processing system to defend against piracy of entertainment software and copyrighted graphics arts. In accordance with the presented model, graphics content providers distribute encrypted 3D graphics data along with their certified licenses. During rendering, when encrypted graphics data, e.g. geometry or textures, are fetched by a digital rights enabled graphics processing system, it will be decrypted. The graphics processing system also ensures that graphics data such as geometry, textures or shaders are bound only in accordance with the binding constraints designated in the licenses. Special API extensions for media/software developers are also proposed to enable our protection model. We evaluated the proposed hardware system based on cycle-based GPU simulator with configuration in line with realistic implementation and open source video game Quake 3D.

Abstract: View-dependent multiresolution rendering places a heavy load on CPU. This paper presents a new method on view-dependent refinement of multiresolution meshes by using the computation power of modern programmable graphics hardware (GPU). Two rendering passes using this method are included. During the first pass, the level of detail selection is performed in the fragment shaders. The resultant buffer from the first pass is taken as the input texture to the second rendering pass by vertex texturing, and then the node culling and triangulation can be performed in the vertex shaders. Our approach can generate adaptive meshes in real-time, and can be fully implemented on GPU. The method improves the efficiency of mesh simplification, and significantly alleviates the computing load on CPU.

Abstract: Radial Basis Functions (RBFs) have become a popular rendering primitive, both in surface and in volume rendering. This paper focuses on volume visualization, giving rise to 3D kernels. RBFs are especially convenient for the representation of scattered and irregularly distributed point samples, where the RBF kernel is used as a blending function for the space in between samples. Common representations employ radially symmetric RBFs, and various techniques have been introduced to render these, also with efficient implementations on programmable graphics hardware (GPUs). In this paper, we extend the existing work to more generalized, ellipsoidal RBF kernels, for the rendering of scattered volume data. We devise a post-shaded kernel-centric rendering approach, specifically designed to run efficiently on GPUs, and we demonstrate our renderer using datasets from subdivision volumes and computational science.