Abstract: Three-dimensional (3D) medical images of computed tomographic (CT) data sets can be generated with a variety of computer algorithms. The three most commonly used techniques are shaded surface display, maximum intensity projection, and, more recently, 3D volume rendering. Implementation of 3D volume rendering involves volume data management, which relates to operations including acquisition, resampling, and editing of the data set; rendering parameters including window width and level, opacity, brightness, and percentage classification; and image display, which comprises techniques such as "fly-through" and "fly-around," multiple-view display, obscured structure and shading depth cues, and kinetic and stereo depth cues. An understanding of both the theory and method of 3D volume rendering is essential for accurate evaluation of the resulting images. Three-dimensional volume rendering is useful in a wide variety of applications but is just now being incorporated into commercially available software packages for medical imaging. Although further research is needed to determine the efficacy of 3D volume rendering in clinical applications, with wider availability and improved cost-to-performance ratios in computing, 3D volume rendering is likely to enjoy widespread acceptance in the medical community.

Abstract: In this paper we present a method for recovering the reflectance properties of all surfaces in a real scene from a sparse set of photographs, taking into account both direct and indirect illumination The result is a lighting-independent model of the scene’s geometry and reflectance properties, which can be rendered with arbitrary modifications to structure and lighting via traditional rendering methods Our technique models reflectance with a lowparameter reflectance model, and allows diffuse albedo to vary arbitrarily over surfaces while assuming that non-diffuse characteristics remain constant across particular regions The method’s input is a geometric model of the scene and a set of calibrated high dynamic range photographs taken with known direct illumination The algorithm hierarchically partitions the scene into a polygonal mesh, and uses image-based rendering to construct estimates of both the radiance and irradiance of each patch from the photographic data The algorithm computes the expected location of specular highlights, and then analyzes the highlight areas in the images by running a novel iterative optimization procedure to recover the diffuse and specular reflectance parameters for each region Lastly, these parameters are used in constructing high-resolution diffuse albedo maps for each surface The algorithm has been applied to both real and synthetic data, including a synthetic cubical room and a real meeting room Rerenderings are produced using a global illumination system under both original and novel lighting, and with the addition of synthetic objects Side-by-side comparisons show success at predicting the appearance of the scene under novel lighting conditions CR Categories: I210 [Artificial Intelligence]: Vision and Scene Understanding—modeling and recovery of physical attributes I37 [Computer Graphics]: Three-dimensional Graphics and Realism—color, shading, shadowing, and texture I37 [Computer Graphics]: Three-Dimensional Graphics and Realism— Radiosity I48 [Image Processing]: Scene Analysis—Color, photometry, shading

Abstract: This paper describes VolumePro, the world’s first single-chip realtime volume rendering system for consumer PCs. VolumePro implements ray-casting with parallel slice-by-slice processing. Our discussion of the architecture focuses mainly on the rendering pipeline and the memory organization. VolumePro has hardware for gradient estimation, classification, and per-sample Phong illumination. The system does not perform any pre-processing and makes parameter adjustments and changes to the volume data immediately visible. We describe several advanced features of VolumePro, such as gradient magnitude modulation of opacity and illumination, supersampling, cropping and cut planes. The system renders 500 million interpolated, Phong illuminated, composited samples per second. This is sufficient to render volumes with up to 16 million voxels (e.g., 256) at 30 frames per second. CR Categories: B.4.2 [Hardware]: Input/Output and Data Communications—Input/Output DevicesImage display; C.3 [Computer Systems Organization]: Special-Purpose and ApplicationBased Systems—Real-time and embedded systems; I.3.1 [Computer Graphics]: Hardware Architecture—Graphics processor;

Abstract: Conventional projector-based display systems are typically designed around precise and regular configurations of projectors and display surfaces. While this results in rendering simplicity and speed, it also means painstaking construction and ongoing maintenance. In previously published work, we introduced a vision of projector-based displays constructed from a collection of casually-arranged projectors and display surfaces. In this paper, we present flexible yet practical methods for realizing this vision, enabling low-cost mega-pixel display systems with large physical dimensions, higher resolution, or both. The techniques afford new opportunities to build personal 3D visualization systems in offices, conference rooms, theaters, or even your living room. As a demonstration of the simplicity and effectiveness of the methods that we continue to perfect, we show in the included video that a 10-year old child can construct and calibrate a two-camera, two-projector, head-tracked display system, all in about 15 minutes.

Abstract: A rendering is an abstraction that favors, preserves, or even emphasizes some qualities while sacrificing, suppressing, or omitting other characteristics that are not the focus of attention. Most computer graphics rendering activities have been concerned with photorealism, i.e., trying to emulate an image that looks like a highquality photograph. This laudable goal is useful and appropriate in many applications, but not in technical illustration where elucidation of structure and technical information is the preeminent motivation. This calls for a different kind of abstraction in which technical communication is central, but art and appearance are still essential instruments toward this end. Work that has been done on computer generated technical illustrations has focused on static images, and has not included all of the techniques used to hand draw technical illustrations. A paradigm for the display of technical illustrations in a dynamic environment is presented. This display environment includes all of the benefits of computer generated technical illustrations, such as a clearer picture of shape, structure, and material composition than traditional computer graphics methods. It also includes the three-dimensional interactive strength of modern display systems. This is accomplished by using new algorithms for real time drawing of silhouette curves, algorithms which solve a number of the problems inherent in previous methods. We incorporate current non-photorealistic lighting methods, and augment them with new shadowing algorithms based on accepted techniques used by artists and studies carried out in human perception. This paper, all of the images, and a mpeg video clip are available at http://www.cs.utah.edu/ bgooch/ITI/. CR Categories: I.3.0 [Computer Graphics]: General; I.3.6 [Computer Graphics]: Methodology and Techniques.

Abstract: With fast 3D graphics becoming more and more available even on low end platforms, the focus in hardware-accelerated rendering is beginning to shift towards higher quality rendering and additional functionality instead of simply higher performance implementations based on the traditional graphics pipeline. In this paper we present techniques for realistic shading and lighting using computer graphics hardware. In particular, we discuss multipass methods for high quality local illumination using physically-based reflection models, as well as techniques for the interactive visualization of non-diffuse global illumination solutions. These results are then combined with normal mapping for increasing the visual complexity of rendered images. Although the techniques presented in this paper work at interactive frame rates on contemporary graphics hardware, we also discuss some modifications of the rendering pipeline that help to further improve both performance and quality of the proposed methods. CR Categories: I.3.1 [Computer Graphics]: Hardware Architecture—Graphics processors; I.3.3 [Computer Graphics]: Picture/Image Generation—Bitmap and frame buffer operations; I.3.6 [Computer Graphics]: Methodology and Techniques— Standards I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Color, Shading, Shadowing and Texture

Abstract: High contrast images are common in night scenes and other scenes that include dark shadows and bright light sources. These scenes are difficult to display because their contrasts greatly exceed the range of most display devices for images. As a result, the image constrasts are compressed or truncated, obscuring subtle textures and details. Humans view and understand high contrast scenes easily, “adapting” their visual response to avoid compression or truncation with no apparent loss of detail. By imitating some of these visual adaptation processes, we developed methods for the improved display of high-contrast images. The first builds a display image from several layers of lighting and surface properties. Only the lighting layers are compressed, drastically reducing contrast while preserving much of the image detail. This method is practical only for synthetic images where the layers can be retained from the rendering process. The second method interactively adjusts the displayed image to preserve local contrasts in a small “foveal” neighborhood. Unlike the first method, this technique is usable on any image and includes a new tone reproduction operator. Both methods use a sigmoid function for contrast compression. This function has no effect when applied to small signals but compresses large signals to fit within an asymptotic limit. We demonstrate the effectiveness of these approaches by comparing processed and unprocessed images.

Abstract: A separable decomposition of bidirectional reflectance distributions (BRDFs) is used to implement arbitrary reflectances from point sources on existing graphics hardware. Two-dimensional texture mapping and compositing operations are used to reconstruct samples of the BRDF at every pixel at interactive rates. A change of variables, the Gram-Schmidt halfangle/difference vector parameterization, improves separability. Two decomposition algorithms are also presented. The singular value decomposition (SVD) minimizes RMS error. The normalized decomposition is fast and simple, using no more space than what is required for the final representation.

Abstract: Finding and displaying silhouette edges is important in applications ranging from computer vision to nonphotorealistic rendering. To render visible silhouette edges of a polygonal object in a scene from a given viewpoint, we must first find all silhouette edges, i.e. boundaries between adjacent front facing and back-facing surfaces. This is followed by solving the partial visibility problem so that only those parts of silhouette edges, which are not occluded by interior of any front facing surface, are rendered. The scene may optionally be rendered with a lighting model. This paper describes a simple general-purpose method to combine all three operations for any scene composed of objects that can be scan-converted. Using a depth buffer, the rendering process computes the intersection of adjacent front facing and back-facing surfaces in image space at interactive rates. All operations are performed in image-precision and hence special care is taken for the limited numerical precision of the depth buffer. A solution is suggested using view- dependent modification of polygonal objects. The method does not require any preprocessing or adjacency information and hence is applicable for dynamic scenes. Our method is based on the observation that only two sets of primitives are needed to compute silhouette edges, the first and second layer of polygons from a given viewpoint. In other words, these include the visible polygons and the layer just behind them which may be either front facing or back facing. Intersection of these two sets gives silhouette edges. In this paper we propose a method specifically for polygonal models but it can be extended to models in any representation for which these two layers or their intersection can be computed. In the proposed method the two nearest layers and their intersection are computed in real time in image precision. The method is robust, general purpose and easy to implement. No pre-processing or adjacency information is required. This allows use of silhouette edges in dynamic scenes, models with changing topology or in models with different levels of detail using traditional polygon rendering pipeline. We also describe how this method can be used to create interesting effects.

Abstract: To create an effective illusion of virtual objects coexisting with the real world, see-through HMD-based Augmented Reality techniques supplement the user's view with images of virtual objects. We introduce here a new paradigm, Spatially Augmented Reality (SAR), where virtual objects are rendered directly within or on the user's physical space. A key benefit of SAR is that the user does not need to wear a head-mounted display. Instead, with the use of spatial displays, wide field of view and possibly high-resolution images of virtual objects can be integrated directly into the environment. For example, the virtual objects can be realized by using digital light projectors to "paint" 2D/3D imagery onto real surfaces, or by using built-in flat panel displays. In this paper we present the rendering method used in our implementation and discuss the fundamentally different visible artifacts that arise as a result of errors in tracker measurements. Finally, we speculate about how SAR techniques might be combined with see-through AR to provide an even more compelling AR experience.

Abstract: The present invention provides a method, system, and computer program product for reflection space image based rendering of an object at an interactive frame rate. A set of source radiance environment maps associated with a set of source viewing vectors are warped to create a destination radiance environment map associated with a destination viewing vector in a current frame. Blending and weighting operations can also be applied in creating the final destination radiance environment map. An object is then rendered with texture environment mapped from the destination radiance environment map.

Abstract: This paper describes a new method for superimposing virtual objects with correct shadings onto an image of a real scene. Unlike the previously proposed methods, our method can measure a radiance distribution of a real scene automatically and use it for superimposing virtual objects appropriately onto a real scene. First, a geometric model of the scene is constructed from a pair of omnidirectional images by using an omnidirectional stereo algorithm. Then, radiance of the scene is computed from a sequence of omnidirectional images taken with different shutter speeds and mapped onto the constructed geometric model. The radiance distribution mapped onto the geometric model is used for rendering virtual objects superimposed onto the scene image. As a result, even for a complex radiance distribution, our method can superimpose virtual objects with convincing shadings and shadows cast onto the real scene. We successfully tested the proposed method by using real images to show its effectiveness.

Abstract: Recent advances in the technology of 3D sensors and in the performance of numerical simulations result in the generation of volume data at ever growing size. In order to allow real-time exploration of even the highest resolution data sets, adaptive techniques benefiting from the hierarchical nature of multi-resolution representations have gained special attention. In this paper we propose an adaptive approach to the fast reconstruction of iso-surfaces from regular volume data at arbitrary levels of detail. The algorithm has been designed to enable real-time navigation through complex structures while providing user-adjustable resolution levels. Since adaptive on-the-fly reconstruction and rendering is performed from a hierarchical octree representation of the volume data, the method does not depend on pre-processing with respect to a specific iso-value thus allowing the user to interactively browse through the set of all possible iso-surfaces. Special attention is paid to the fixing of cracks in the surface where the adaptive reconstruction level changes and to the efficient estimation of the iso-surface’s curvature.

Abstract: Overview and perspective registration and rendering - image-based approach multi sensory augmentation communication and collaboration systems - design issues and future.

Abstract: Recent interest in large displays has led to renewed development of tiled displays, which are comprised of several individual displays arranged in an array and used as one large logical display. Stanford's “Interactive Mural” is an example of such a display, using an overlapping four by two array of projectors that back-project onto a diffuse screen to form a 6' by 2' display area with a resolution of over 60 dpi. Writing software to make effective use of the large display space is a challenge because normal window system interaction metaphors break down. One promising approach is to switch to immersive applications; another approach, the one we are investigating, is to emulate office, conference room or studio environments which use the space to display a collection of visual material to support group activities.In this paper we describe a virtual graphics system that is designed to support multiple simultaneous rendering streams from both local and remote sites. The system abstracts the physical number of computers, graphics subsystems and projectors used to create the display. We provide performance measurements to show that the system scales well and thus supports a variety of different hardware configurations. The system is also interesting because it uses transparent “layers,” instead of windows, to manage the screen.

Abstract: Three dimensional scenes are typically modeled using a single, fixed resolution model of each geometric object. Renderings of such a model are often either slow or crude, however: slow for distant objects, where the chosen detail level is excessive, and crude for nearby objects, where the detail level is insufficient. What is needed is a multiresolution model that represents objects at multiple levels of detail. With a multiresolution model, a rendering program can choose the level of detail appropriate for the object’s screen size so that less time is wasted drawing insignificant detail. The principal challenge is the development of algorithms that take a detailed model as input and automatically simplify it, while preserving appearance. Multiresolution techniques can be used to speed many applications, including real time rendering for architectural and terrain simulators, and slower, higher quality rendering for entertainment and radiosity. This paper surveys existing multiresolution modeling techniques and speculates about what might be possible in the future.

Abstract: Apparata and methods for rapid design of objects/shapes in Computer-Aided Design (CAD) tools and in Virtual Reality (VR) environments are described. The underlying geometric representation of the objects within the design tool is optimized so that design activities such as modeling, editing, rendering, etc. can be processed extremely rapidly, thereby enhancing the response time of the design tool. The representation is preferably provided in two parts, which may be referred to as a “design intent model” and a “shape model”. The design intent model is a higher-level representation wherein elements are arranged in hierarchical parent-child relationships which record the elements' assembly sequence. The shape model is a lower-level representation storing more detailed information about the elements and their relationships. During editing of the design, the user acts on the design intent model, and the design intent model is mapped to the shape design model so that it is updated to reflect the changes therein. The design intent model is in many cases sufficient by itself to allow basic editing of the model and rendering of the edited model, but where editing operations grow sufficiently complex that the design intent model lacks sufficient information to allow the operation to be performed, the shape model can be relied upon for the information necessary to complete the operation.

Abstract: A system for visualizing, controlling and managing information includes a data analysis unit for interpreting and classifying raw data using analytical techniques. A data flow coordination unit routes data from its source to other components within the system. A data preparation unit handles the graphical preparation of the data and a data rendering unit presents the data in a three-dimensional interactive environment where the user can observe, interact with, and interpret the data. A user can view the information on various levels, from a high overall process level view, to a view illustrating linkage between variables, to view the hard data itself, or to view results of an analysis of the data. The system allows a user to monitor a physical process in real-time and further allows the user to manage and control the information in a manner not previously possible.

Abstract: Selective quality light field operations efficiently manipulate a multi-resolution representation of a light field. These operations include intra-image and inter-image decomposition and compression of a light field to a multi-resolution representation. These operations also include intra-image and inter-image decompression and reconstruction of a light field at selective quality. These selective quality operations also apply to storage, rendering, and transmission. Various techniques improve spatial displacement estimation of a prediction light field image from a reference light field image. These techniques includes constraining the placement and size of a search window based upon a geometrical relationship between prediction and reference light field images, hierarchical spatial displacement estimation, edge extension of a reference light field image, differential coding of displacement vectors, and multi-predictor spatial displacement estimation. Configuring reference and prediction light field images in view of geometrical relationships between light field images also improves spatial displacement estimation.

Abstract: An image generator is organized into a plurality of rendering engines, each of which renders an image of a part scene and provides the part image to a merge engine associated with that rendering engine. The image is a part image in that it usually contains less than all of the objects in the image to be rendered. The merge engine merges the part image from its associated rendering engine with the part image provided by a prior merge engine and provides the merged part image to a next merge engine. One or more merge engines are designated the output merge engines and these output merge engines output a merged part image that is (a portion of) the ultimate output of the image generator, the full rendered image. Each merge engine performs its merge process on the pixels it has from its rendering engine and from its prior neighbor merge engine, in a pipelined manner and without necessarily waiting for all of the pixels of the part image or the merged part image.

Abstract: An apparatus and method for real-time volume processing and universal three-dimensional rendering (10). The apparatus includes a plurality of three-dimensional memory units; at least one pixel bus for providing global horizontal communication; a plurality of rendering pipelines; at least one geometry bus; and a control unit. The plurality of rendering pipelines each preferably include hardware for interpolation, shading, FIFO buffering, communication and lookup tables. The apparatus of the present invention may be coupled to a geometry pipeline (18) for mixing surfaces, images and volumes together in a single image. A method for performing volumetric ray casting of a 3D volume includes the steps of calculating a distance along a major projection axis from a predefined viewpoint; dividing the volume into a plurality of consecutive regions having exponentially increasing bounds; casting a plurality of rays from the viewpoint through the volume; either merging two or more rays or splitting one or more rays at the region boundaries; and repeating the ray casting and merging/splitting steps until the entire volume has been processed. The apparatus and methods of the present invention achieve true real-time performance for high-resolution volume rendering, mixing surfaces and volumes in a single image operations, including texture mapping and image-based rendering.

Abstract: A system and method for seamlessly combining client-only rendering techniques with server-only rendering techniques. The approach uses a composite stream containing three distinct streams. Two of the streams are synchronized and transmit camera definition, video of server-rendered objects, and a time dependent depth map for the server-rendered object. The third stream is available to send geometry from the server to the client, for local rendering if appropriate. The invention can satisfy a number of viewing applications. For example, initially the most relevant geometry can stream to the client for high quality local rendering while the server delivers renderings of less relevant geometry at lower resolutions. After the most relevant geometry has been delivered to the client, the less important geometry can be optionally streamed to the client to increase the fidelity of the entire scene. In the limit, all of the geometry is transferred to the client and the situation corresponds to client-only rendering system where local graphics hardware is used to improve fidelity and reduce bandwidth. Alternatively, if a client does not have local three-dimensional graphics capability then the server can transmit only the video of the server-rendered object and drop the other two streams. In either case, the approach also permits for a progressive improvement in the server-rendered image whenever the scene becomes static. Bandwidth that was previously used to represent changing images is allocated to improving the fidelity of the server-rendered image whenever the scene becomes static.

Abstract: Multi-projector systems are increasingly being used to provide large-scale and high-resolution displays for next-generation interactive 3D graphics applications, including large-scale data visualization, immersive virtual environments, and collaborative design. These systems must include a very high-performance and scalable 3D rendering subsystem in order to generate high-resolution images at real-time frame rates. This paper describes a sort-first parallel rendering system for a scalable display wall system built with a network of PCs, graphics accelerators, and portable projectors. The main challenge is to develop scalable algorithms to partition and assign rendering tasks effectively under the performance and functionality constraints of system area networks, PCs, and commodity 3-D graphics accelerators. We have developed three coarse-grained partitioning algorithms, incorporated them into a working prototype system, and run initial experiments aimed at evaluating algorithmic trade-offs and performance bottlenecks in such a system. Results of our experiments indicate that the coarse-grained characteristics of the sort-first architecture are well suited for constructing a parallel rendering system running on a PC cluster.

Abstract: An apparatus and methods for rendering 3D-graphics images preferably includes a port (850) for receiving commands from a graphics application, an output (234) for sending a rendered image to a display and a geometry-operations pipeline, coupled to the port and to the output (234), the geometry-operations pipeline including a block for performing transformations (614). In one embodiment, the block for performing transformations includes a co-extensive logical and first physical stages, as well as a second physical stage including multiple logical stages. The second physical stage includes multiple logical stages that interleave their execution.

Abstract: This thesis investigates the use of stereo vision for the application of view synthesis. View synthesis--the problem of creating images of a scene as it would appear from novel viewpoints--has traditionally been approached using methods from computer graphics. These methods, however, suffer from low rendering speed, limited achievable realism, and, most severely, their dependence on a global scene model, which typically needs to be constructed manually. In this thesis, we present a new approach to view synthesis that avoids the above problems by synthesizing new views from existing images of a scene. Using an image-based representation of scene geometry computed by stereo vision methods, a global model can be avoided, and realistic new views can be synthesized quickly using image warping. The new application of stereo for view synthesis makes it necessary to re-evaluate the requirements on stereo algorithms. We compare view synthesis to several traditional applications of stereo, and conclude that stereo vision is better suited for view synthesis than for applications requiring explicit 3D reconstruction. We also discuss ways of dealing with partially occluded regions of unknown depth and with completely occluded regions of unknown texture, and present experiments demonstrating that it is possible to efficiently synthesize realistic new views even from inaccurate and incomplete depth information. This thesis also contributes several novel stereo algorithms that are motivated by the specific requirements imposed by view synthesis. We introduce a new evidence measure based on intensity gradients for establishing correspondences between images. This measure combines the notions of similarity and confidence, and allows stable matching and easy assigning of canonical depth interpretations in image regions of insufficient information. We also present new diffusion-based stereo algorithms that are motivated by the need to correctly recover object boundaries. In particular, we develop a novel Bayesian estimation technique that significantly outperforms area-based algorithms using fixed-sized windows. We provide experimental results for all algorithms on both synthetic and real images.

Abstract: A method of reducing the dynamic range of an input image for effectively rendering the input image on an output display medium, the method comprises the steps of extracting detail signals and coarse signals from the input image; generating contrast gain-control signals from the input image by detecting coarse scale edges; modifying the detail signals according to the contrast gain signals; and, adding the modified detail signals to the coarse signals to obtain an output image

Abstract: Rather than implementing a printer as a single set of rendering parameters, a printer can be implemented as a “meta-printer” that comprises one or more virtual printers. The virtual printers define different sets of rendering parameters. Each virtual printer is accessed through a page description language or through a user interface presiding in a print driver or server. In particular, each virtual printer of the meta-printer has an associated set of rendering parameters. One virtual printer of this meta-printer may be set up to print a first type of content using a first set of selected rendering parameters. This same virtual printer may be set up to print a different type of content using a different set of selected rendering parameters. This may be repeated for any number of additional content types. A user can access these virtual printers to print a document containing a variety of different content types. A user may select one virtual printer of the meta-printer to render an entire document. In this case, if the selected virtual printer defines only a single selected rendering parameter option for each rendering parameter, that rendering parameter option will be used to render all of the different various content types in that document. In contrast, if the selected virtual printer defines different selected rendering parameter options for different content types for various rendering parameters, the different content types will be rendered differently using the particular rendering parameter options selected for each content type.

Abstract: We present a new technique which enables direct volume rendering based on 3D texture mapping hardware, enabling shading as well as classification of the interpolated data. Our technique supports accurate lighting for a one directional light source, semi-transparent classification, and correct blending. To circumvent the limitations of one general classification, we introduce multiple classification spaces which are very valuable to understand the visualized data, and even mandatory to comprehensively grasp the 3D relationship of different materials present in the volumetric data. Furthermore, we illustrate how multiple classification spaces can be realized using existing graphics hardware.In contrast to previously reported algorithms, our technique is capable of performing all the above mentioned tasks within the graphics pipeline. Therefore, it is very efficient: The three dimensional texture needs to be stored only once and no load is put onto the CPU. Besides using standard OpenGL functionality, we exploit advanced per pixel operations and make use of available OpenGL extensions.