Abstract: We introduce ideas, proposed technologies, and initial results for an office of the future that is based on a unified application of computer vision and computer graphics in a system that combines and builds upon the notions of the CAVE™, tiled display systems, and image-based modeling . The basic idea is to use real-time computer vision techniques to dynamically extract per-pixel depth and reflectance information for the visible surfaces in the office including walls, furniture, objects, and people, and then to either project images on the surfaces, render images of the surfaces , or interpret changes in the surfaces. In the first case, one could designate every-day (potentially irregular) real surfaces in the office to be used as spatially immersive display surfaces, and then project high-resolution graphics and text onto those surfaces. In the second case, one could transmit the dynamic image-based models over a network for display at a remote site. Finally, one could interpret dynamic changes in the surfaces for the purposes of tracking, interaction, or augmented reality applications. To accomplish the simultaneous capture and display we envision an office of the future where the ceiling lights are replaced by computer controlled cameras and “smart” projectors that are used to capture dynamic image-based models with imperceptible structured light techniques, and to display high-resolution images on designated display surfaces. By doing both simultaneously on the designated display surfaces, one can dynamically adjust or autocalibrate for geometric, intensity, and resolution variations resulting from irregular or changing display surfaces, or overlapped projector images. Our current approach to dynamic image-based modeling is to use an optimized structured light scheme that can capture per-pixel depth and reflectance at interactive rates. Our system implementation is not yet imperceptible, but we can demonstrate the approach in the laboratory. Our approach to rendering on the designated (potentially irregular) display surfaces is to employ a two-pass projective texture scheme to generate images that when projected onto the surfaces appear correct to a moving headtracked observer. We present here an initial implementation of the overall vision, in an office-like setting, and preliminary demonstrations of our dynamic modeling and display techniques.

Abstract: Radiance is a collection of approximately 50 programs that do everything from object modeling to point calculation, rendering, image processing and display. This is the definitive reference on the radiance lighting simulation and rendering system.

Abstract: We present a new method for creating an image with a handpainted appearance from a photograph, and a new approach to designing styles of illustration. We “paint” an image with a series of spline brush strokes. Brush strokes are chosen to match colors in a source image. A painting is built up in a series of layers, starting with a rough sketch drawn with a large brush. The sketch is painted over with progressively smaller brushes, but only in areas where the sketch differs from the blurred source image. Thus, visual emphasis in the painting corresponds roughly to the spatial energy present in the source image. We demonstrate a technique for painting with long, curved brush strokes, aligned to normals of image gradients. Thus we begin to explore the expressive quality of complex brush strokes. Rather than process images with a single manner of painting, we present a framework for describing a wide range of visual styles. A style is described as an intuitive set of parameters to the painting algorithm that a designer can adjust to vary the style of painting. We show examples of images rendered with different styles, and discuss long-term goals for expressive rendering styles as a general-purpose design tool for artists and animators. CR

Abstract: This paper presents how the image-based rendering technique of view-dependent texture-mapping (VDTM) can be efficiently implemented using projective texture mapping, a feature commonly available in polygon graphics hardware. VDTM is a technique for generating novel views of a scene with approximately known geometry making maximal use of a sparse set of original views. The original presentation of VDTM in by Debevec, Taylor, and Malik required significant per-pixel computation and did not scale well with the number of original images. In our technique, we precompute for each polygon the set of original images in which it is visible and create a "view map" data structure that encodes the best texture map to use for a regularly sampled set of possible viewing directions. To generate a novel view, the view map for each polygon is queried to determine a set of no more than three original images to blended together in order to render the polygon with projective texture-mapping. Invisible triangles are shaded using an object-space hole-filling method. We show how the rendering process can be streamlined for implementation on standard polygon graphics hardware. We present results of using the method to render a large-scale model of the Berkeley bell tower and its surrounding campus enironment.

Abstract: Although direct volume rendering is a powerful tool for visualizing complex structures within volume data, the size and complexity of the parameter space controlling the rendering process makes generating an informative rendering challenging. In particular, the specification of the transfer function-the mapping from data values to renderable optical properties-is frequently a time consuming and unintuitive task. Ideally, the data being visualized should itself suggest an appropriate transfer function that brings out the features of interest without obscuring them with elements of little importance. We demonstrate that this is possible for a large class of scalar volume data, namely that where the regions of interest are the boundaries between different materials. A transfer function which makes boundaries readily visible can be generated from the relationship between three quantities: the data value and its first and second directional derivatives along the gradient direction. A data structure we term the histogram volume captures the relationship between these quantities throughout the volume in a position independent, computationally efficient fashion. We describe the theoretical importance of the quantities measured by the histogram volume, the implementation issues in its calculation, and a method for semiautomatic transfer function generation through its analysis. We conclude with results of the method on both idealized synthetic data as well as real world datasets.

Abstract: We present an integral equation which generalizes a variety of known rendering algorithms. In the course of discussing a monte carlo solution we also present a new form of variance reduction, called Hierarchical sampling and give a number of elaborations shows that it may be an efficient new technique for a wide variety of monte carlo procedures. The resulting rendering algorithm extends the range of optical phenomena which can be effectively simulated.

Abstract: OpenGLand its extensionsprovide accessto advancedper-pixel operationsavailable in the rasterizationstageand in the frame buffer hardware of modern graphicsworkstations. With these mechanisms, completelynew renderingalgorithmscanbedesigned andimplementedin a very particularway. In this paperwe extend theideaof extensi vely usinggraphicshardwarefor therenderingof volumetricdatasetsin variousways. First, we introducethe conceptof clipping geometriesby meansof stencilbuffer operations, and we exploit pixel texturesfor the mappingof volume datato sphericaldomains.We show waysto use3D texturesfor the renderingof lightedandshadediso-surfacesin real-timewithout extractingany polygonalrepresentation. Second,wedemonstratethat even for volumedataon unstructuredgrids, whereonly software solutionsexist up to now, bothmethods,iso-surfaceextractionand directvolumerendering,canbeacceleratedto new ratesof interactivity by simplepolygondrawing andframebuffer operations. CR Categories: I.3.7 [ComputerGraphics]:Three-Dimensional GraphicsandRealism—Graphics Hardware, 3D Textures, Volume Rendering, Unstructured Grids

Abstract: A technique, system, and computer program for using servlets to isolate the retrieval of data from the rendering of the data into a presentation format. Data retrieval logic is isolated to a data servlet, and presentation formatting is isolated to a rendering servlet. Servlet chaining is used to send the output of the data servlet to the rendering servlet. The data servlet formats its output data stream for transfer to a downstream servlet. This data stream may be formatted using a language such as the Extensible Markup Language (XML), according to a specific Document Type Definition (DTD). The rendering servlet parses this XML data stream, using a style sheet that may be written using the Extensible Style Language (XSL), and creates a HyperText Markup Language (HTML) data stream as output.

Abstract: In this paper we introduce a new compressed representation for the connectivity of a triangle mesh. We present local compression and decompression algorithms which are fast enough for real time applications. The achieved space compression rates keep pace with the best rates reported for any known global compression algorithm. These nice properties have great benefits for several important applications. Naturally, the technique can be used to compress triangle meshes without significant delay before they are stored on external devices or transmitted over a network. The presented decompression algorithm is very simple allowing a possible hardware realization of the decompression algorithm which could significantly increase the rendering speed of pipelined graphics hardware. CR Categories: I.3.1 [Computer Graphics]: Hardware Architecture; I.3.3 [Computer Graphics]: Picture/Image Generation— Display algorithms

Abstract: Real-time rendering of triangulated surfaces has attracted growing interest in the last few years. However, interactive visualization of very large scale grid digital elevation models is still difficult. The graphics load must be controlled by adaptive surface triangulation and by taking advantage of different levels of detail. Furthermore, management of the visible scene requires efficient access to the terrain database. We describe an all-in-one visualization system which integrates adaptive triangulation, dynamic scene management and spatial data handling. The triangulation model is based on the restricted quadtree triangulation. Furthermore, we present new algorithms of restricted quadtree triangulation. These include among others exact error approximation, progressive meshing, performance enhancements and spatial access.

Abstract: Alain Fournier University of Toronto Don Fussell The University of Texas at Austin Loren Carpenter Lucasfilm the techniques used to implement the model. We introduce a new algorithm that computes a realistic, visually satisfactory approximation to fractional Brownian m o tion in faster time than with exact calculations. A major advantage of this technique is that it allows us to compute the surface to arbitrary levels of details without increasing the database. Thus objects with complex appearances can be displayed from a very small database. The character of the surface can be controlled by merely modifying a few parameters. A similar change allows complex motion to be created inexpensively.

Abstract: Real-time rendering of triangulated surfaces has attracted growing interest in the last few years. However, interactive visualization of very large scale grid digital elevation models is still difficult. The graphics load must be controlled by adaptive surface triangulation and by taking advantage of different levels of detail. Furthermore, management of the visible scene requires efficient access to the terrain database. We describe an all-in-one visualization system which integrates adaptive triangulation, dynamic scene management and spatial data handling. The triangulation model is based on the restricted quadtree triangulation. Furthermore, we present new algorithms of restricted quadtree triangulation. These include among others exact error approximation, progressive meshing, performance enhancements and spatial access.

Abstract: We describe the clipmap, a dynamic texture representation that efficiently caches textures of arbitrarily large size in a finite amount of physical memory for rendering at real-time rates. Further, we describe a software system for managing clipmaps that supports integration into demanding real-time applications. We show the scale and robustness of this integrated hardware/software architecture by reviewing an application virtualizing a 170 gigabyte texture at 60 Hertz. Finally, we suggest ways that other rendering systems may exploit the concepts underlying clipmaps to solve related problems. CR

Abstract: A program and method for a map display tool for use with a navigation system and used with a map database. The map display tool is adapted to render a map on a display or in a graphics buffer for a predetermined geographic region. The map display tool includes a map processing layer for performing the essential data manipulation, a graphics interface layer that maintains a series of look up tables with graphics commands, and a primitive layer customized to interact with an underlying graphics platform and the graphics interface layer. The map display tool renders a map by retrieving data items from the map database and processing each data item. Shape information is rendered on a screen or in a graphics buffer, name information is buffered and sorted to optimize filtering, prioritization, spatial conflicts resolution and other tasks prior to rendering.

Abstract: Creating realistic images has been a major focus in the study of computer graphics for much of its history. This effort has led to mathematical models and algorithms that can compute predictive, or physically realistic, images from known camera positions and scene descriptions that include the geometry of objects, the reflectance of surfaces, and the lighting used to illuminate the scene. These images accurately describe the physical quantities that would be measured from a real scene. Because these algorithms can predict real images, they can also be used in inverse problems to work backward from photographs to attributes of the scene. Work on three such inverse rendering problems is described. The first, inverse lighting, assumes knowledge of geometry, reflectance, and the recorded photograph and solves for the lighting in the scene. A technique using a linear least-squares system is proposed and demonstrated. Also demonstrated is an application of inverse lighting, called re-lighting, which modifies lighting in photographs. The second two inverse rendering problems solve for unknown reflectance, given images with known geometry, lighting, and camera positions. Photographic texture measurement concentrates on capturing the spatial variation in an object's reflectance. The resulting system begins with scanned 3D models of real objects and uses photographs to construct accurate, high-resolution textures suitable for physically realistic rendering. The system is demonstrated on two complex natural objects with detailed surface textures. Image-based BRDF measurement takes the opposite approach to reflectance measurement, capturing the directional characteristics of a surface's reflectance by measuring the bidirectional reflectance distribution function, or BRDF. Using photographs of an object with spatially uniform reflectance, the BRDFs of paints and papers are measured with completeness and accuracy that rival that of measurements obtained using specialized devices. The image-based approach and novel light source positioning technique require only general-purpose equipment, so the cost of the apparatus is low compared to conventional approaches. In addition, very densely sampled data can be measured very quickly, when the wavelength spectrum of the BRDF does not need to be measured in detail.

Abstract: A pre-processing method prepares 3D objects for rendering to image layers in a layered graphics rendering pipeline. The method transforms a bounding volume for an object to a 2D bounding box in a 2D view space. It then subdivides the bounding box into blocks of image samples called chunks. To computer the portion of the object that should be rendered to each chunk, it determines which geometric primitives of the object overlap each chunk and generates a list of primitives to the rendered for each chunk.

Abstract: High quality rendering and physics based modeling in volume graphics have been limited because intensity based volumetric data do not represent surfaces well. High spatial frequencies due to abrupt intensity changes at object surfaces result in jagged or terraced surfaces in rendered images. The use of a distance-to-closest-surface function to encode object surfaces is proposed. This function varies smoothly across surfaces and hence can be accurately reconstructed from sampled data. The zero value iso surface of the distance map yields the object surface and the derivative of the distance map yields the surface normal. Examples of rendered images are presented along with a new method for calculating distance maps from sampled binary data.

Abstract: A virtual view system (10) uses raw imagery from cameras (17) placed around a sporting arena to generate virtual views of the sporting event from any contemplated view point. The system (10) consists of an optical tracking system (11), a virtual environment server (12), and one or more virtual view stations (13). The optical tracking system (11) receives raw, 2-dimensional video data from a pre-selected number of cameras (17) strategically placed around a sporting arena. The body state data is then passed to a virtual environment server (12) which generates body position information and visual models for transfer to a selected number of virtual view stations (13). Each virtual view station (13) includes viewing software for rendering and viewing a virtual sports scene as desired. The view stations (13) also control the viewing point of view of a virtual camera and output video data to a video production center (128) so that video data may be transmitted and combined with other video input as needed. Optional subsystems such as a control center (14) and an archive computer (16) may be integrated into the system to alter camera positioning, tether, focus, and zoom, and to store processed data for the sporting event and replay the data on demand.

Abstract: Most current image-based rendering methods operate under the assumption that all of the visible surfaces in the scene are opaque ideal diffuse (Lambertian) reflectors. This paper is concerned with image-based rendering of non-diffuse synthetic scenes. We introduce a new family of image-based scene representations and describe corresponding image-based rendering algorithms that are capable of handling general synthetic scenes containing not only diffuse reflectors, but also specular and glossy objects. Our image-based representation is based on layereddepth images. It represents simultaneously and separately both view-independent scene information and view-dependent appearance information. The view-dependent information may be either extracted directly from our data-structures, or evaluated procedurally using an image-based analogue of ray tracing. We describe image-based rendering algorithms that recombine the two components together in a manner that produces a good approximation to the correct image from any viewing position. In addition to extending image-based rendering to non-diffuse synthetic scenes, our paper has an important methodological contribution: it places image-based rendering, light field rendering, and volume graphics in a common framework of discrete raster-based scene representations.

Abstract: An adaptable image segmentation system for differentially rendering black and white and/or color images using a plurality of imaging techniques. An image is segmented according to classes of regions that may be rendered according to the same imaging techniques. Image regions may be rendered according to a three class system (such as traditional text, graphic and picture systems), or according to more than three image classes. In addition, only two image classes may be required to render high quality draft or final output images. The image characteristics that may be rendered differently from class to class may include halftoning, colorization and other image attributes.

Abstract: We propose a general framework for the simulation of sounds produced by colliding physical objects in a virtual reality environment. The framework is based on the vibration dynamics of bodies. The computed sounds depend on the material of the body, its shape, and the location of the contact. This simulation of sounds allows the user to obtain important auditory clues about the objects in the simulation, as well as about the locations on the objects of the collisions. Specifically, we show how to compute (1) the spectral signature of each body (its natural frequencies), which depends on the material and the shape, (2) the “timbre” of the vibration (the relative amplitudes of the spectral components) generated by an impulsive force applied to the object at a grid of locations, (3) the decay rates of the various frequency components that correlate with the type of material, based on its internal friction parameter, and finally (4) the mapping of sounds onto the object's geometry for real-time rendering of the resulting sound. The framework has been implemented in a Sonic Explorer program which simulates a room with several objects such as a chair, tables, and rods. After a preprocessing stage, the user can hit the objects at different points to interactively produce realistic sounds.

Abstract: Graphics software for use in rendering an image on a computer display. The software is used in conjunction with special hardware including a high resolution computer monitor and high speed graphics imaging software. The software allows the user to create and manipulate paint objects that define the way images are modified by rendering tools such as a drawing tool or a titling tool. The attributes for a plurality of paint objects are defined in a hierarchy and can be stored on a memory device such as a computer hard drive. The organization into a hierarchy of the various attributes making up a complex object such as a paint stroke or a text body result in an ability to deal with branches of the hierarchy as persistable objects in their own right.

Abstract: Image-based or light field rendering has received much recent attention as an alternative to traditional geometric methods for modeling and rendering complex objects. A light field represents the radiance flowing through all the points in a scene in all possible directions. We explore two new techniques for efficiently acquiring, storing, and reconstructing light fields in a (nearly) uniform fashion. Both techniques sample the light field by sampling the set of lines that intersect a sphere tightly fit around a given object. Our first approach relies on uniformly subdividing the sphere and representing this subdivision in a compact data structure which allows efficient mapping of image pixels or rays to sphere points and then to subdivision elements. We sample a light field by joining pairs of subdivision elements and store the resulting samples in a multi-resolution, highly compressed fashion that allows efficient rendering. Our second method allows a uniform sampling of all five dimensions of the light field, using hierarchical subdivision for directional space and uniform grid sampling for positional space. Light field models are acquired using parallel projections along a set of uniform directions. Depth information can also be stored for high-quality image rendering. The system can provide bounds on key sources of error in the representation and can be generalized to arbitrary scenes comprising multiple complex objects.

Abstract: A system is provided for reconstruction the 3-dimensional motions of a human figure from a monocularly viewed image sequence in which a statistical approach is used coupled with the use of a set of motion capture examples to build a gaussian probability model for short human motion sequences. In a simplified rendering domain, this yields an optimal 3-d estimate of human motion used to create realistic animations given a 2-d temporal sequence. The subject system is also useful to identify which motion modes are difficult to estimate. In one embodiment, a stick figure is overlaid on the input video image to allow manual correction of incorrectly tracked body parts. The utilization of stick figure correction permits manual correction for a more accurate 3-d reconstruction of the motion depicted in the input video sequence. An interactive tracking system processes real video sequences, and achieves good 3-d reconstructions of the human figure motion.

Abstract: A robust, semi-automatic way is presented to create 3D polyhedral building models from computer-drawn floor plans, requiring minimal user interaction. The models adhere to a consistent solids representation and can be used for computer rendering, visualization in interactive walkthroughs, and in various simulation and analysis programs. The output of our prototype program is directly compatible with the Berkeley WALKTHRU system and with the NIST CFAST fire simulator. A consistent model of a seven-storey building with more than 300 rooms has been generated in the time span of a few days from original AutoCAD floor plans drawn by architects.

Abstract: Many sophisticated solutions have been proposed to reduce the geometric complexity of 3D meshes A problem studied less often is how to preserve on a simplified mesh the detail (eg, color, high frequency shape detail, scalar fields, etc) which is encoded in the original mesh We present a general approach for preserving detail on simplified meshes The detail (or high frequency information) lost after simplification is encoded through texture or bump maps The original contribution is that preservation is performed after simplification, by building set of triangular texture patches that are then packed in a single texture map Each simplified mesh face is sampled to build the associated triangular texture patch; a new method for storing this set of texture patches into a standard rectangular texture is presented and discussed Our detail preserving approach makes no assumptions about the simplification process adopted to reduce mesh complexity and allows highly efficient rendering The solution is very general, allowing preservation of any attribute value defined on the high resolution mesh We also describe an alternative application: the conversion of 3D models with 3D static procedural textures into standard 3D models with 2D textures

Abstract: In the scope of rendering complex models with high depth complexity, it is of great importance to design output-sensitive algorithms, i.e., algorithms with the time complexity proportional to the number of visible graphic primitives in the resulting image. In this paper an algorithm allowing efficient culling of the invisible portion of the rendered model is presented. Our approach uses a spatial hierarchy to represent the topology of the model. For a current viewpoint a set of polygonal occluders is determined that are used to build the occlusion tree. In the occlusion tree occlusion volumes of the selected occluders are merged. Visibility from the viewpoint is determined by processing the spatial hierarchy and classifying the visibility of its regions. In this process the occlusion tree is used to determine the viewpoint-to-region visibility efficiently. The algorithm is well-suited for complex models where large occluders are present.