Abstract: The NVIDIA® OptiX™ ray tracing engine is a programmable system designed for NVIDIA GPUs and other highly parallel architectures. The OptiX engine builds on the key observation that most ray tracing algorithms can be implemented using a small set of programmable operations. Consequently, the core of OptiX is a domain-specific just-in-time compiler that generates custom ray tracing kernels by combining user-supplied programs for ray generation, material shading, object intersection, and scene traversal. This enables the implementation of a highly diverse set of ray tracing-based algorithms and applications, including interactive rendering, offline rendering, collision detection systems, artificial intelligence queries, and scientific simulations such as sound propagation. OptiX achieves high performance through a compact object model and application of several ray tracing-specific compiler optimizations. For ease of use it exposes a single-ray programming model with full support for recursion and a dynamic dispatch mechanism similar to virtual function calls.

Abstract: Current imaging methods such as Magnetic Resonance Imaging (MRI), Confocal microscopy, Electron Microscopy (EM) or Selective Plane Illumination Microscopy (SPIM) yield three-dimensional (3D) data sets in need of appropriate computational methods for their analysis. The reconstruction, segmentation and registration are best approached from the 3D representation of the data set. Here we present a platform-independent framework based on Java and Java 3D for accelerated rendering of biological images. Our framework is seamlessly integrated into ImageJ, a free image processing package with a vast collection of community-developed biological image analysis tools. Our framework enriches the ImageJ software libraries with methods that greatly reduce the complexity of developing image analysis tools in an interactive 3D visualization environment. In particular, we provide high-level access to volume rendering, volume editing, surface extraction, and image annotation. The ability to rely on a library that removes the low-level details enables concentrating software development efforts on the algorithm implementation parts. Our framework enables biomedical image software development to be built with 3D visualization capabilities with very little effort. We offer the source code and convenient binary packages along with extensive documentation at http://3dviewer.neurofly.de .

Abstract: The color rendering index (CRI) has been shown to have deficiencies when applied to white light-emitting-diode-based sources. Furthermore, evidence suggests that the restricted scope of the CRI unnecessarily penalizes some light sources with desirable color qualities. To solve the problems of the CRI and include other dimensions of color quality, the color quality scale (CQS) has been developed. Although the CQS uses many of elements of the CRI, there are a number of fundamental differences. Like the CRI, the CQS is a test-samples method that compares the appearance of a set of reflective samples when illuminated by the test lamp to their appearance under a reference illuminant. The CQS uses a larger set of reflective samples, all of high chroma, and combines the color differences of the samples with a root mean square. Additionally, the CQS does not penalize light sources for causing increases in the chroma of object colors but does penalize sources with smaller rendered color gamut areas. The scale of the CQS is converted to span 0-100, and the uniform object color space and chromatic adaptation transform used in the calculations are updated. Supplementary scales have also been developed for expert users.

Abstract: Optimizing operation of a media player during rendering of media files. The invention includes authoring software to create a data structure and to populate the created data structure with obtained metadata. The invention also includes rendering software to retrieve the metadata from the data structure and to identify media files to render. In one embodiment, the invention is operable as part of a compressed media format having a set of small files containing metadata, menus, and playlists in a compiled binary format designed for playback on feature-rich personal computer media players as well as low cost media players.

Abstract: Plenoptic cameras, constructed with internal microlens arrays, capture both spatial and angular information, i.e., the full 4-D radiance, of a scene. The design of traditional plenoptic cameras assumes that each microlens image is completely defocused with respect to the image created by the main camera lens. As a result, only a single pixel in the final image is rendered from each microlens image, resulting in disappointingly low resolution. A recently devel- oped alternative approach based on the focused plenoptic camera uses the microlens array as an imaging system focused on the im- age plane of the main camera lens. The flexible spatioangular trade- off that becomes available with this design enables rendering of final images with significantly higher resolution than those from traditional plenoptic cameras. We analyze the focused plenoptic camera in optical phase space and present basic, blended, and depth-based rendering algorithms for producing high-quality, high-resolution im- ages. We also present our graphics-processing-unit-based imple- mentations of these algorithms, which are able to render full screen refocused images in real time. © 2010 SPIE and IS&T.

Abstract: This state of the art report covers reconstruction methods for transparent and specular objects or phenomena. While the 3D acquisition of opaque surfaces with Lambertian reflectance is a well-studied problem, transparent, refractive, specular and potentially dynamic scenes pose challenging problems for acquisition systems. This report reviews and categorizes the literature in this field. Despite tremendous interest in object digitization, the acquisition of digital models of transparent or specular objects is far from being a solved problem. On the other hand, real-world data is in high demand for applications such as object modelling, preservation of historic artefacts and as input to data-driven modelling techniques. With this report we aim at providing a reference for and an introduction to the field of transparent and specular object reconstruction. We describe acquisition approaches for different classes of objects. Transparent objects/phenomena that do not change the straight ray geometry can be found foremost in natural phenomena. Refraction effects are usually small and can be considered negligible for these objects. Phenomena as diverse as fire, smoke, and interstellar nebulae can be modelled using a straight ray model of image formation. Refractive and specular surfaces on the other hand change the straight rays into usually piecewise linear ray paths, adding additional complexity to the reconstruction problem. Translucent objects exhibit significant sub-surface scattering effects rendering traditional acquisition approaches unstable. Different classes of techniques have been developed to deal with these problems and good reconstruction results can be achieved with current state-of-the-art techniques. However, the approaches are still specialized and targeted at very specific object classes. We classify the existing literature and hope to provide an entry point to this exiting field.

Abstract: Some embodiments of the invention provide a mobile device that captures and produces images with high dynamic ranges. To capture and produce a high dynamic range image, the mobile device of some embodiments includes novel image capture and processing modules. In some embodiments, the mobile device produces a high dynamic range (HDR) image by (1) having its image capture module rapidly capture a succession of images at different image exposure durations, and (2) having its image processing module composite these images to produce the HDR image.

Abstract: We present an algorithm designed for navigating around a performance that was filmed as a "casual" multi-view video collection: real-world footage captured on hand held cameras by a few audience members. The objective is to easily navigate in 3D, generating a video-based rendering (VBR) of a performance filmed with widely separated cameras. Casually filmed events are especially challenging because they yield footage with complicated backgrounds and camera motion. Such challenging conditions preclude the use of most algorithms that depend on correlation-based stereo or 3D shape-from-silhouettes.Our algorithm builds on the concepts developed for the exploration of photo-collections of empty scenes. Interactive performer-specific view-interpolation is now possible through innovations in interactive rendering and offline-matting relating to i) modeling the foreground subject as video-sprites on billboards, ii) modeling the background geometry with adaptive view-dependent textures, and iii) view interpolation that follows a performer. The billboards are embedded in a simple but realistic reconstruction of the environment. The reconstructed environment provides very effective visual cues for spatial navigation as the user transitions between viewpoints. The prototype is tested on footage from several challenging events, and demonstrates the editorial utility of the whole system and the particular value of our new inter-billboard optimization.

Abstract: Acquiring and representing the 4D space of rays in the world (the light field) is important for many computer vision and graphics applications. Yet, light field acquisition is costly due to their high dimensionality. Existing approaches either capture the 4D space explicitly, or involve an error-sensitive depth estimation process. This paper argues that the fundamental difference between different acquisition and rendering techniques is a difference between prior assumptions on the light field. We use the previously reported dimensionality gap in the 4D light field spectrum to propose a new light field prior. The new prior is a Gaussian assigning a non-zero variance mostly to a 3D subset of entries. Since there is only a low-dimensional subset of entries with non-zero variance, we can reduce the complexity of the acquisition process and render the 4D light field from 3D measurement sets. Moreover, the Gaussian nature of the prior leads to linear and depth invariant reconstruction algorithms. We use the new prior to render the 4D light field from a 3D focal stack sequence and to interpolate sparse directional samples and aliased spatial measurements. In all cases the algorithm reduces to a simple spatially invariant deconvolution which does not involve depth estimation.

Abstract: Systems and methods are provided for the organization, management, and graphical display of multiple logically related entities. In some embodiments, the systems and methods provide the real-time display of status and location information for a fleet of tracked vehicles. In some embodiments, the systems and methods permit the automatic clustering of assets based on a user's view of an underlying map. In some embodiments, the assets are vehicles that are a part of a managed fleet. In some embodiments, the assets are logically arranged into clusters of like assets. In some embodiments, the clusters provide graphical indications of status or class information of their underlying assets.

Abstract: A computer network communication method and system wherein software rendering software is interposed in the data communication path between a browser running on a user computer and the internet data sources (for example, internet-accessible server computers) that the user browser wants to receive information from. The software rendering application gets data from internet data sources, but this data may contain malware. To provide enhanced security, the software rendering application renders this data to form a new browser readable code set (for example, an xml page with CSS layers), and this new and safe browser readable code set is sent along to the browser on the user computer for appropriate presentation to the user. As part of the rendering process, dedicated and distinct virtual machines may be used to render certain portion of the data, such as executable code. These virtual machines may be watched, and quickly destroyed if it is detected that they have encountered some type of malware.

Abstract: For more than three decades, computer graphics researchers have been working to create a photorealistic digital actor. In the Digital Emily project, Image Metrics and the University of Southern California's Institute for Creative Technologies (USC ICT) animated a digital face using new results in 3D facial capture, character modeling, animation, and rendering. The project aimed to cross the "uncanny valley" that divides a synthetic-looking face from a real, animated, expressive person. The key technologies included a fast high-resolution digital face-scanning process using USC ICT's Light Stage capture system and Image Metrics' video-based facial-animation system. The project generated one of the first photorealistic digital faces to speak and emote convincingly in a medium close-up.LED lights turned on. We've described the Light Stage scanning process elsewhere. Unlike earlier Light Stage processes, which photographed the face under hundreds of illumination directions, this newer process requires only 15 stereo photographs of the face under different lighting conditions to capture the face's geometry and reflectance. The small number of photos lets us use off-the-shelf high-quality still cameras (Canon ID Mark Ill's) and shoot each data set in under three seconds using the cameras' burst mode.

Abstract: A graphical user interface, system and method are described controlling a video stream. For example, a data processing apparatus according to one embodiment has a memory for storing program code and a processor for processing the program code to generate a graphical user interface (“GUI”) comprising: a plurality of video thumbnails arranged in a pattern, each of the video thumbnails rendering a compressed version of a real-time video of a particular movie, television series, game, or other type of video content; and a selection graphic highlighting a currently-selected video thumbnail, the thumbnails moving within the GUI in relation to the selection graphic in response to user input from an input device.

Abstract: Audio playback is used to enhance the control of Personal Digital Entertainment devices. Embodiments of the present invention provide audio message files associated with the activation of a control or provide other device status messaging Audio messages are also provided in file sets that are user selectable to give the user a choice of ‘themes’.

Abstract: This paper introduces a new scalable technique for approximating indirect illumination in fully dynamic scenes for real-time applications, such as video games. We use lattices and spherical harmonics to represent the spatial and angular distribution of light in the scene. Our technique does not require any precomputation and handles large scenes with nested lattices. It is primarily targeted at rendering single-bounce indirect illumination with occlusion, but can be extended to handle multiple bounces and participating media. We demonstrate that our method produces plausible results even when running on current game console hardware with a budget of only a few milliseconds for performing all computation steps for indirect lighting. We evaluate our technique and show it in combination with a variety of popular real-time rendering techniques.

Abstract: Localization microscopy techniques based on localizing single fluorophore molecules now routinely achieve accuracies better than 30 nm. Unlike conventional optical microscopies, localization microscopy experiments do not generate an image but a list of discrete coordinates of estimated fluorophore positions. Data display and analysis therefore generally require visualization methods that translate the position data into conventional images. Here we investigate the properties of several widely used visualization techniques and show that a commonly used algorithm based on rendering Gaussians may lead to a 1.44-fold loss of resolution. Existing methods typically do not explicitly take sampling considerations into account and thus may produce spurious structures. We present two additional visualization algorithms, an adaptive histogram method based on quad-trees and a Delaunay triangulation based visualization of point data that address some of these deficiencies. The new visualization methods are designed to suppress erroneous detail in poorly sampled image areas but avoid loss of resolution in well-sampled regions. A number of criteria for scoring visualization methods are developed as a guide for choosing among visualization methods and are used to qualitatively compare various algorithms.

Abstract: An exemplary method includes accessing cardiac information acquired via a catheter located at various positions in a venous network of a heart of a patient where the cardiac information comprises position information, electrical information and mechanical information; mapping local electrical activation times to anatomic positions to generate an electrical activation time map; mapping local mechanical activation times to anatomic positions to generate a mechanical activation time map; generating an electromechanical delay map by subtracting local electrical activation times from corresponding local mechanical activation times; and rendering at least the electromechanical delay map to a display Various other methods, devices, systems, etc, are also disclosed

Abstract: Methods, apparatus, and computer-readable storage media for simulating artistic effects in images rendered from plenoptic data. An impressionistic-style artistic effect may be generated in output images of a rendering process by an “impressionist” 4D filter applied to the microimages in a flat captured with focused plenoptic camera technology. Individual pixels are randomly selected from blocks of pixels in the microimages, and only the randomly selected pixels are used to render an output image. The randomly selected pixels are rendered to generate the artistic effect, such as an “impressionistic” effect, in the output image. A rendering technique is applied that samples pixel values from microimages using a thin sampling kernel, for example a thin Gaussian kernel, so that pixel values are sampled only from one or a few of the microimages.

Abstract: Lack of accurate focus cues in conventional stereoscopic displays has potentially significant effects on depth perception accuracy and visual fatigue. Recently several multi-focal plane display prototypes have been demonstrated with the promise of improving the accuracy of focus cue rendering in stereoscopic displays. In this paper, we present a systematic method to address two fundamental issues in designing a multi-focal plane display: (1) the appropriate dioptric spacing between adjacent focal planes; and (2) the depth-weighted fusing function to render a continuous three-dimensional (3-D) volume using a sparse number of focal planes placed in the space. By taking account of both ocular factors of the human visual system (HVS) and display factors of a multi-focal plane system, we determine that an appropriate spacing between two adjacent focal planes should be ~0.6 diopter (D) while a smaller spacing may be necessary for further improving retinal image quality. We further develop a set of nonlinear depth-weighted fusing function with the promise of balancing perceptual continuity of a 3-D scene and retinal image quality. Our method was based on quantitative evaluation of the modulation transfer functions (MTF) of depth-fused images formed on retina.

Abstract: Methods, apparatus, and computer-readable storage media for rendering focused plenoptic camera data. A depth-based rendering technique is described that estimates depth at each microimage and then applies that depth to determine a position in the input flat from which to read a value to be assigned to a given point in the output image. The techniques may be implemented according to parallel processing technology that renders multiple points of the output image in parallel. In at least some embodiments, the parallel processing technology is graphical processing unit (GPU) technology.

Abstract: A tile-based graphics processing pipeline that uses primitive lists that can encompass plural rendering tiles includes a primitive list reading unit that reads primitive lists for a tile being rendered to determine primitives to be processed for the tile and a rasteriser that rasterises input primitives to generate graphics fragments to be processed. The pipeline further comprises a comparison unit between the primitive list reading unit and the rasteriser that for primitives that have been read from primitive lists that include plural rendering tiles, compares the location of the primitive in the render target to the location of the tile being rendered, and then either sends the primitive onwards to the rasteriser if the comparison determines that the primitive could lie at least partially within the tile, or does not send the primitive to the rasteriser if the comparison determines that the primitive definitely does not lie within the tile.

Abstract: A super-resolved demosaicing technique for rendering focused plenoptic camera data performs simultaneous super-resolution and demosaicing. The technique renders a high-resolution output image from a plurality of separate microimages in an input image at a specified depth of focus. For each point on an image plane of the output image, the technique determines a line of projection through the microimages in optical phase space according to the current point and angle of projection determined from the depth of focus. For each microimage, the technique applies a kernel centered at a position on the current microimage intersected by the line of projection to accumulate, from pixels at each microimage covered by the kernel at the respective position, values for each color channel weighted according to the kernel. A value for a pixel at the current point in the output image is computed from the accumulated values for the color channels.

Abstract: The present disclosure describes various techniques that may be implemented to execute and/or interpret manipulation gestures performed by a user on a multipoint touch input interface of a computing device. An example method includes receiving a multipoint touch gesture at a multipoint touch input interface of a computing device, wherein the multipoint touch gesture comprises a gesture that is performed with multiple touches on the multipoint touch input interface, and resolving the multipoint touch gesture into a command. The example method further includes determining at least one physical simulation effect to associate with the resolved multipoint touch gesture, and rendering a unified feedback output action in a graphical user interface of the computing device by executing the command, wherein the unified feedback output action includes at least a graphical output action incorporated with the at least one physical simulation effect in the graphical user interface.

Abstract: A system for creating and editing image and or text-based projects includes a server connected to a data network, the server having access to a digital medium containing an executable graphics user interface (GUI) accessible by computer browser, at least one data repository residing on the network, the repository accessible to the server, the repository containing images, and product-related graphics, a print configuration engine implemented in software residing on and executable from the digital medium or another accessible medium, the print configuration engine for rendering image and or text-based products for printing, and at least one bi-directional communication channel established between a computing appliance running a version of the GUI while connected to the network and a server managing the data repository.

Abstract: A method and system for providing computer-generated output and in particular graphical output. The system includes a network configured to carry digital information. The system includes a server in communication with the network, the server configured to execute an application and a cloud engine module. The application provides a graphical output. The output capturing and encoding engine module is further configured to intercept the graphical output from the application on the server. The output capturing and encoding engine module is further configured to convert the graphical output into at least one of: graphical commands and video codec data. The output capturing and encoding engine module is further configured to transmit the converted output over the network. The system includes a client in communication with the server over the network, the client configured to execute a graphics and video decoding and rendering engine module. The graphics and video decoding and rendering engine module is configured to, responsive to receiving the transmitted converted output, rendering the graphical output. The graphics and video decoding and rendering engine module is configured to intercept graphics and video decoding and rendering inputs at the client. The graphics and video decoding and rendering engine module is configured to transmit the intercepted user inputs to the output capturing and encoding engine module.

Abstract: Although direct volume rendering is established as a powerful tool for the visualization of volumetric data, efficient and reliable feature detection is still an open topic. Usually, a tradeoff between fast but imprecise classification schemes and accurate but time-consuming segmentation techniques has to be made. Furthermore, the issue of uncertainty introduced with the feature detection process is completely neglected by the majority of existing approaches.In this paper we propose a guided probabilistic volume segmentation approach that focuses on the minimization of uncertainty. In an iterative process, our system continuously assesses uncertainty of a random walker-based segmentation in order to detect regions with high ambiguity, to which the user's attention is directed to support the correction of potential misclassifications. This reduces the risk of critical segmentation errors and ensures that information about the segmentation's reliability is conveyed to the user in a dependable way. In order to improve the efficiency of the segmentation process, our technique does not only take into account the volume data to be segmented, but also enables the user to incorporate classification information. An interactive workflow has been achieved by implementing the presented system on the GPU using the OpenCL API. Our results obtained for several medical data sets of different modalities, including brain MRI and abdominal CT, demonstrate the reliability and efficiency of our approach.

Abstract: In this paper, the new challenges of 3DTV for subjective assessment are discussed Conventional 2D methods have severe limitations which will be revealed Based on the understanding of the new characteristics brought by 3DTV, changes and additions in the requirements for subjective assessment are proposed in order to develop a novel subjective video quality assessment methodology for 3DTV In particular, depth rendering for 3D display is selected to give a further discussion The depth rendering abilities are defined as a combination of the physical parameters and the perceptual constrains We analyze different types of stereoscopic and multiview displays Several problems regarding depth rendering are discussed in order to highlight the diversity and complexity of assessing 3DTV

Abstract: The radiative transfer framework that underlies all current rendering of volumes is limited to scattering media whose properties are invariant to rotation. Many systems allow for "anisotropic scattering," in the sense that scattered intensity depends on the scattering angle, but the standard equation assumes that the structure of the medium is isotropic. This limitation impedes physics-based rendering of volume models of cloth, hair, skin, and other important volumetric or translucent materials that do have anisotropic structure. This paper presents an end-to-end formulation of physics-based volume rendering of anisotropic scattering structures, allowing these materials to become full participants in global illumination simulations.We begin with a generalized radiative transfer equation, derived from scattering by oriented non-spherical particles. Within this framework, we propose a new volume scattering model analogous to the well-known family of microfacet surface reflection models; we derive an anisotropic diffusion approximation, including the weak form required for finite element solution and a way to compute the diffusion matrix from the parameters of the scattering model; and we also derive a new anisotropic dipole BSSRDF for anisotropic translucent materials. We demonstrate results from Monte Carlo, finite element, and dipole simulations. All these contributions are readily implemented in existing rendering systems for volumes and translucent materials, and they all reduce to the standard practice in the isotropic case.

Abstract: Although three-dimensional (3D) displays enhance visual quality more than two-dimensional (2D) displays do, the depth information required for 3D displays is unavailable in the conventional 2D content. Therefore, converting 2D videos into 3D ones has become an important issue in emerging 3D applications. This work presents a novel algorithm that automatically converts 2D videos into 3D ones. The proposed algorithm utilizes the edge information to segment the image into object groups. A depth map is then assigned based on a hypothesized depth gradient model. Next, the depth map is block-based assigned by cooperating with a cross bilateral filter to generate visually comfortable depth maps efficiently and also diminish the block artifacts. A multiview video can be readily generated by using a depth image-based rendering method.