Abstract: We describe an image based rendering approach that generalizes many current image based rendering algorithms, including light field rendering and view-dependent texture mapping. In particular, it allows for lumigraph-style rendering from a set of input cameras in arbitrary configurations (i.e., not restricted to a plane or to any specific manifold). In the case of regular and planar input camera positions, our algorithm reduces to a typical lumigraph approach. When presented with fewer cameras and good approximate geometry, our algorithm behaves like view-dependent texture mapping. The algorithm achieves this flexibility because it is designed to meet a set of specific goals that we describe. We demonstrate this flexibility with a variety of examples.

Abstract: We consider the rendering of diffuse objects under distant illumination, as specified by an environment map. Using an analytic expression for the irradiance in terms of spherical harmonic coefficients of the lighting, we show that one needs to compute and use only 9 coefficients, corresponding to the lowest-frequency modes of the illumination, in order to achieve average errors of only 1%. In other words, the irradiance is insensitive to high frequencies in the lighting, and is well approximated using only 9 parameters. In fact, we show that the irradiance can be procedurally represented simply as a quadratic polynomial in the cartesian components of the surface normal, and give explicit formulae. These observations lead to a simple and efficient procedural rendering algorithm amenable to hardware implementation, a prefiltering method up to three orders of magnitude faster than previous techniques, and new representations for lighting design and image-based rendering.

Abstract: ed Al Bovic San Diego: Academic Press (2000) 891pp, price: £64.95, ISBN: 0 12 119790 5 If you want to know anything about image and video processing, this book is the place to begin and, probably, to end as well. Encyclopaedic in scale, 58 chapters by 97 contributors are packed into 891 pages. The subject is divided into 10 sections: an introduction, basic image processing techniques, image and video processing, image and video analysis, image compression, video compression, image and video acquisition, image and video rendering and assessment, image and video storage retrieval and communications, and applications of image processing. The boundaries of the coverage of the subject are quite sharply defined. First of all, an image is taken to mean a still picture that does not change with time, whereas a video evolves with time and generally contains moving and/or changing objects. There are two chapters on image and video acquisition; these are concerned with image capture, Fourier analysis, sampling rate, interpolation and reconstruction of continuous time-varying imagery. Neither image display nor perception are mentioned, even in the index. Image fusion is not included. Thus, the book is strictly limited to image and video processing and that is certainly not a criticism. As far as the depth of coverage is concerned, the objective is ambitious. This is to provide a resource that covers introductory, intermediate and advanced topics with equal clarity. I tested the extent to which this has been achieved by trying to find the answers to some questions about image and video compression. In the 277 pages of the 10 chapters devoted to this topic, there is a wealth of information extending from lossless coding, through other coding and quantization schemes, wavelet and JPEG compression, video and object-based coding, to MPEG video standards. I learned enough to realize that the subject is so mature, and the investment in both hardware and software so significant, that there must be little to be gained from research at the margins of any new approaches. It is the final section (134 pages) that contains the chapters that will be of most immediate interest to readers with a medical bias. This section is concerned with applications of image processing. The relevant chapters are on: computed tomography (x-ray, CT, PET and SPECT) with the emphasis on image reconstruction and three-dimensional topics; cardiac image processing, including the analysis of cardiac mechanics, perfusion and (perhaps oddly) electrocardiography; computer aided detection for screening mammography, concentrating on masses, calcifications and segmentation; and confocal microscopy. It is actually in these chapters that most information is to be found concerning the instrumentation for image acquisition. This is not an expensive book. Indeed, it is stupendously good value-for-money. Where else, in the field of image and video processing, can the knowledge of 97 contributors, all clearly expert and many deservedly famous, be so conveniently accessed? If there is a criticism, it has to be that the 55 pages of colour illustrations (which are additional to the 891 pages of text) are gathered together in four blocks, distributed through the book. These illustrations also appear in the text, but in grey scale. So, it is sometimes necessary to turn to the colour sections to appreciate the points that are being made. But this is only a small nuisance in exchange for what is presumably a substantial economy in the price of the book. Realistically, this is too big a book to be read from cover to cover. Consequently, it needs a good index and there is no criticism here. For example, I looked up 'ultrasound imaging' and found a reference to intravascular scanning and its combination with biplane angiography to create a map of the vessel in 3D space; this is a technique of which I was previously unaware. I also found a reference to echography, with an informative discussion of segmentation which, in this situation, is a far from trivial task. This handbook is the first in a new Academic Press series in communications, networking and multimedia, with Jerry Gibson of Southern Methodist University as the editor-in-chief. In editing and co-authoring the handbook, Al Bovik of the University of Texas at Austin has discharged a monumental assignment with spectacular success. If you are interesting in image and video processing, you must have a copy.

Abstract: Modern laser range and optical scanners need rendering techniques that can handle millions of points with high resolution textures. This paper describes a point rendering and texture filtering technique called surface splatting which directly renders opaque and transparent surfaces from point clouds without connectivity. It is based on a novel screen space formulation of the Elliptical Weighted Average (EWA) filter. Our rigorous mathematical analysis extends the texture resampling framework of Heckbert to irregularly spaced point samples. To render the points, we develop a surface splat primitive that implements the screen space EWA filter. Moreover, we show how to optimally sample image and procedural textures to irregular point data during pre-processing. We also compare the optimal algorithm with a more efficient view-independent EWA pre-filter. Surface splatting makes the benefits of EWA texture filtering available to point-based rendering. It provides high quality anisotropic texture filtering, hidden surface removal, edge anti-aliasing, and order-independent transparency.

Abstract: The paper addresses the problem of "class-based" image-based recognition and rendering with varying illumination. The rendering problem is defined as follows: Given a single input image of an object and a sample of images with varying illumination conditions of other objects of the same general class, re-render the input image to simulate new illumination conditions. The class-based recognition problem is similarly defined: Given a single image of an object in a database of images of other objects, some of them multiply sampled under varying illumination, identify (match) any novel image of that object under varying illumination with the single image of that object in the database. We focus on Lambertian surface classes and, in particular, the class of human faces. The key result in our approach is based on a definition of an illumination invariant signature image which enables an analytic generation of the image space with varying illumination. We show that a small database of objects-in our experiments as few as two objects-is sufficient for generating the image space with varying illumination of any new object of the class from a single input image of that object. In many cases, the recognition results outperform by far conventional methods and the re-rendering is of remarkable quality considering the size of the database of example images and the mild preprocess required for making the algorithm work.

Abstract: Realism in computer-generated images requires accurate input models for lighting, textures and BRDFs. One of the best ways of obtaining high-quality data is through measurements of scene attributes from real photographs by inverse rendering. However, inverse rendering methods have been largely limited to settings with highly controlled lighting. One of the reasons for this is the lack of a coherent mathematical framework for inverse rendering under general illumination conditions. Our main contribution is the introduction of a signal-processing framework which describes the reflected light field as a convolution of the lighting and BRDF, and expresses it mathematically as a product of spherical harmonic coefficients of the BRDF and the lighting. Inverse rendering can then be viewed as deconvolution. We apply this theory to a variety of problems in inverse rendering, explaining a number of previous empirical results. We will show why certain problems are ill-posed or numerically ill-conditioned, and why other problems are more amenable to solution. The theory developed here also leads to new practical representations and algorithms. For instance, we present a method to factor the lighting and BRDF from a small number of views, i.e. to estimate both simultaneously when neither is known.

Abstract: We introduce a novel texture-based volume rendering approach that achieves the image quality of the best post-shading approaches with far less slices. It is suitable for new flexible consumer graphics hardware and provides high image quality even for low-resolution volume data and non-linear transfer functions with high frequencies, without the performance overhead caused by rendering additional interpolated slices. This is especially useful for volumetric effects in computer games and professional scientific volume visualization, which heavily depend on memory bandwidth and rasterization power.We present an implementation of the algorithm on current programmable consumer graphics hardware using multi-textures with advanced texture fetch and pixel shading operations. We implemented direct volume rendering, volume shading, arbitrary number of isosurfaces, and mixed mode rendering. The performance does neither depend on the number of isosurfaces nor the definition of the transfer functions, and is therefore suited for interactive high-quality volume graphics.

Abstract: The ubiquity of computer-generated imagery around us, in movies, advertising or on the Internet is already being taken for granted and what impresses most people is the photorealistic quality of the images. Pictures, as we have often been told, are worth a thousand words and the information transported by an image can take many different forms. Many computer graphics researchers are exploring non-photorealistic rendering techniques as an alternative to realistic rendering. Defined by what it is not, non-photorealistic rendering brings art and science together, concentrating less on the process and more on the communication content of an image. Techniques that have long been used by artists can be applied to computer graphics to emphasize subtle attributes and to omit extraneous information. This book provides an overview of the published research on non-photorealistic rendering in order to categorize and distill the current research into a body of usable techniques. A summary of some of the algorithms as well as pseudo-code for producing some of the images is included.

Abstract: Polygonal models currently dominate interactive computer graphics. This is chiefly because of their mathematical simplicity: polygonal models lend themselves to simple, regular rendering algorithms that embed well in hardware, which has in turn led to widely available polygon rendering accelerators for every platform. Unfortunately, the complexity of these models, which is measured by the number of polygons, seems to grow faster than the ability of our graphics hardware to render them interactively. Put another way, the number of polygons we want always seems to exceed the number of polygons we can afford. Polygonal simplification techniques offer one solution for developers grappling with complex models. These methods simplify the polygonal geometry of small, distant, or otherwise unimportant portions of the model, seeking to reduce the rendering cost without a significant loss in the scene's visual content. The article surveys polygonal simplification algorithms, identifies the issues in picking an algorithm, relates the strengths and weaknesses of different approaches, and describes several published algorithms.

Abstract: Meshes are dominantly used to represent 3D models as they fit well with graphics rendering hardware. Features such as volume, moments, and Fourier transform coefficients need to be calculated from the mesh representation efficiently. We propose an algorithm to calculate these features without transforming the mesh into other representations such as the volumetric representation. To calculate a feature for a mesh, we show that we can first compute it for each elementary shape such as a triangle or a tetrahedron, and then add up all the values for the mesh. The algorithm is simple and efficient, with many potential applications.

Abstract: Route maps, which depict a path from one location to another, have emerged as one of the most popular applications on the Web. Current computer-generated route maps, however, are often very difficult to use. In this paper we present a set of cartographic generalization techniques specifically designed to improve the usability of route maps. Our generalization techniques are based both on cognitive psychology research studying how route maps are used and on an analysis of the generalizations commonly found in handdrawn route maps. We describe algorithmic implementations of these generalization techniques within LineDrive, a real-time system for automatically designing and rendering route maps. Feedback from over 2200 users indicates that almost all believe LineDrive maps are preferable to using standard computer-generated route maps alone.

Abstract: The present invention relates to methods and systems for conducting three-dimensional image analysis and diagnosis and possible treatment relating thereto. The invention includes methods of handling signals containing information (data) relating to three-dimensional representation of objects scanned by a scanning medium. The invention also includes methods of making and analyzing volumetric measurements and changes in volumetric measurements which can be used for the purpose of diagnosis and treatment.

Abstract: A system and method for effectively rendering high dynamic range images may include a rendering manager that initially divides an original luminance image into a plurality of original subband images. The rendering manager may then convert the original subband images into original contrast images which may then be converted into original perceived contrast images. The rendering manager may then perform a compression procedure upon the original perceived contrast images to produce compressed perceived contrast images. The rendering manager may next convert the compressed perceived contrast images into compressed contrast images which may then be converted into compressed subband images. The rendering manager may then perform a subband combination procedure for combining the compressed subband images together with a lowest-frequency subband image to thereby generate a rendered luminance image. The rendering manager may finally combine rendered the rendered luminance image with corresponding chrominance information to generate a rendered composite image that may be utilized in any appropriate manner.

Abstract: We describe WireGL, a system for scalable interactive rendering on a cluster of workstations. WireGL provides the familiar OpenGL API to each node in a cluster, virtualizing multiple graphics accelerators into a sort-first parallel renderer with a parallel interface. We also describe techniques for reassembling an output image from a set of tiles distributed over a cluster. Using flexible display management, WireGL can drive a variety of output devices, from standalone displays to tiled display walls. By combining the power of virtual graphics, the familiarity and ordered semantics of OpenGL, and the scalability of clusters, we are able to create time-varying visualizations that sustain rendering performance over 70,000,000 triangles per second at interactive refresh rates using 16 compute nodes and 16 rendering nodes.

Abstract: This paper presents the ARCHEOGUIDE project (Augmented Reality-based Cultural Heritage On-site GUIDE). ARCHEOGUIDE is an IST project, funded by the EU, aiming at providing a personalized electronic guide and tour assistant to cultural site visitors. The system provides on-site help and Augmented Reality reconstructions of ancient ruins, based on user's position and orientation in the cultural site, and realtime image rendering. It incorporates a multimedia database of cultural material for on-line access to cultural data, virtual visits, and restoration information. It uses multi-modal user interfaces and personalizes the flow of information to its user's profile in order to cater for both professional and recreational users, and for applications ranging from archaeological research, to education, multimedia publishing, and cultural tourism. This paper presents the ARCHEOGUIDE system and the experiences gained from the evaluation of an initial prototype by representative user groups at the archeological site of Olympia, Greece.

Abstract: An apparatus and method for real-time volume processing and universal three-dimensional rendering. The apparatus includes a plurality of three-dimensional (3D) memory units (24); at least one pixel bus for providing global horizontal communication (34); a plurality of rendering pipelines; at least one geometry bus (46); and a control unit (38). The apparatus includes a block processor having a circular ray integration pipeline for processing voxel data and ray data. Rays are generally processed in image order thus permitting great flexibility (e.g., perspective projection, global illumination). The block processor includes a splatting unit and a scattering unit. A method for casting shadows and performing global illumination in relation to light sources, which includes sweeping a two dimensional array of rays through the volume, can also be implemented with the apparatus. A method for approximating a perspective projection includes using parallel projection.

Abstract: A language independent, voice based user interface method includes receiving voice input data spoken by a user, identifying a language spoken by the user from the voice input data, converting the voice input data into a first text in the identified language by recognizing the user's speech in the voice input data based at least in part on the language identifier, parsing the first text to extract a keyword, and using the keyword as a command to an application. Further actions include receiving results to the command, converting the results into a second text in a natural language format according to the identified language, and rendering the second text for perception by the user.

Abstract: This paper presents a method for realistic real-time rendering of clouds suitable for flight simulation and games. It provides a cloud shading algorithm that approximates multiple forward scattering in a preprocess, and first order anisotropic scattering at runtime. Impostors are used to accelerate cloud rendering by exploiting frame-to-frame coherence in an interactive flight simulation. Impostors are shown to be particularly well suited to clouds, even in circumstances under which they cannot be applied to the rendering of polygonal geometry. The method allows hundreds of clouds and hundreds of thousands of particles to be rendered at high frame rates, and improves interaction with clouds by reducing artifacts introduced by direct particle rendering techniques.

Abstract: We consider applications using depth-based image-based rendering (IBR), where the synthesis of arbitrary views occur at a remote location, necessitating the compression and transmission of depth maps. Traditional image compression has been designed to provide maximum perceived visual quality, and a direct application is sub-optimal for depth-map compression, since depth-maps are not directly viewed. In other words, the sensitivity of the rendering error depends on the image content as well as on the depth map, we propose two improvements to take this into account. Firstly, we consider region-of-interest (ROI) coding, where we identify those regions of the image where accurate depth is most crucial. Secondly, we reshape the dynamic range of the depth map. Our experiments show a significant improvement in coding gain (1.1 dB) and rendering quality when we integrated these two improvements into a standard JPEG-2000 coder.

Abstract: We describe a system for constructing computer models of several aspects of physical interaction behavior, by scanning the response of real objects. The behaviors we can successfully scan and model include deformation response, contact textures for interaction with force-feedback, and contact sounds. The system we describe uses a highly automated robotic facility that can scan behavior models of whole objects. We provide a comprehensive view of the modeling process, including selection of model structure, measurement, estimation, and rendering at interactive rates. The results are demonstrated with two examples: a soft stuffed toy which has significant deformation behavior, and a hard clay pot which has significant contact textures and sounds. The results described here make it possible to quickly construct physical interaction models of objects for applications in games, animation, and e-commerce.

Abstract: A method of protecting a digital work uses a format preserving encryption scheme to encrypt the digital work. This enables any native replay application or rendering application to transform an encrypted digital work into encrypted presentation data. The originator's digital content is protected in its original form by not being decrypted. This method enables the rendering or replay application to process the encrypted document into encrypted presentation data without decrypting it first. Encrypted presentation data is then decrypted just before it is displayed to the user. An additive encryption scheme is a particular type of encryption scheme which preserves formatting of a digital work.

Abstract: A system and method of rendering overlapping layers in a computer display, such as a windowing system, employs front-to-back assembly of the displayed image. An arbitrary number of overlapping elements, such as windows, can be presented, without requiring temporary storage space or additional off-screen buffers. The front-to-back assembly technique minimizes the number of memory transfers performed in connection with rendering an image, and avoids unnecessary reading and processing of pixels that will not contribute to the final image. Special effects such as semi-transparency, shadows, and irregular shapes can be accommodated and processed in an efficient manner.

Abstract: We consider the problem of video stabilization: removing unwanted image perturbations due to unstable camera motions. We approach this problem from an image-based rendering (IBR) standpoint. Given an unstabilized video sequence, the task is to synthesize a new sequence as seen from a stabilized camera trajectory. This task is relatively straightforward if one has a Euclidean reconstruction of the unstabilized camera trajectory and a suitable IBR algorithm. However, it is often not feasible to obtain a Euclidean reconstruction from an arbitrary video sequence. In light of this problem, we describe IBR techniques for non-metric reconstructions, which are often much easier to obtain since they do not require camera calibration. These rendering techniques are well suited to the video stabilization problem. The key idea behind our techniques is that all measurements are specified in the image space, rather than in the non-metric space.

Abstract: Apparatus for image-guided surgery includes medical imaging apparatus. The imaging apparatus is utilized for capturing 3-dimensional (3D) volume data of patient portions in reference to a coordination system. A computer processes the volume data so as to provide a graphical representation of the data. A stero camera assembly captures a stereoscopic video view of a scene including at least portions of the patient. A tracking system measures pose data of the stereoscopic video view in reference to the coordinate system. The computer is utilized for rendering the graphical representation and the stereoscopic video view in a blended way in conjunction with the pose data so as to provide a stereoscopic augmented image. A head-mounted video-see-through displays the stereoscopic augmented image.

Abstract: We present a new approach for computing generalized proximity information of arbitrary 2D objects using graphics hardware. Using multi-pass rendering techniques and accelerated distance computation, our algorithm performs proximity queries not only for detecting collisions, but also for computing intersections, separation distance, penetration depth, and contact points and normals. Our hybrid geometry and image-based approach balances computation between the CPU and graphics subsystems. Geometric object-space techniques coarsely localize potential intersection regions or closest features between two objects, and image-space techniques compute the low-level proximity information in these regions. Most of the proximity information is derived from a distance field computed using graphics hardware. We demonstrate the performance in collision response computation for rigid and deformable body dynamics simulations. Our approach provides proximity information at interactive rates for a variety of simulation strategies for both backtracking and penalty-based collision responses.

Abstract: We describe a calibration and rendering technique for a projector that can render rectangular images under keystoned position. The projector utilizes a rigidly attached camera to form a stereo pair. We describe a very easy to use technique for calibration of the projector-camera pair using only black planar surfaces. We present an efficient rendering method to pre-warp images so that they appear correctly on the screen, and show experimental results.

Abstract: A rendering application on a computing device requests a digital rights management (DRM) system on the computing device to authorize digital content rendering based on a corresponding digital license. If unavailable, the DRM system attempts to silently acquire the license from a license server without the intervention of a user. In the course thereof, the rendering application receives status information from the DRM system relating to the attempted license acquisition thereby and displays the received status information in a rendering application status display portion. If silent acquisition fails, the rendering application hosts a browser, causes the browser to navigate to a license server, allows a user to communicate with the license server by way of the hosted browser to acquire the license, and shuts down the hosted browser upon reception of the license from the license server.

Abstract: A server architecture for a digital rights management system that distributes and protects rights in content. The server architecture includes a retail site which sells content items to consumers, a fulfillment site which provides to consumers the content items sold by the retail site, and an activation site which enables consumer reading devices to use content items having an enhanced level of copy protection. Each retail site is equipped with a URL encryption object, which encrypts, according to a secret symmetric key shared between the retail site and the fulfillment site, information that is needed by the fulfillment site to process an order for content sold by the retail site. An activation site provides an activation certificate and a secure repository executable to consumer content-rendering devices which enables those content rendering devices to render content having an enhanced level of copy-resistance. The activation site 'activates' client-reading devices in a way that binds them to a persona, and limits the number of devices that may be activated for a particular persona, or the rate at which such devices may be activated for a particular persona.

Abstract: Integral photography (IP), which is one of the ideal 3-D photographic technologies, can be regarded as a method of capturing and displaying light rays passing through a plane. The NHK Science and Technical Research Laboratories have developed a real-time IP system using an HDTV camera and an optical fiber array. In this paper, the authors propose a method of synthesizing arbitrary views from IP images captured by the HDTV camera. This is a kind of image-based rendering system, founded on the 4-D data space representation of light rays. Experimental results show the potential to improve the quality of images rendered by computer graphics techniques.