Abstract: This paper describes a method for scene reconstruction of complex, detailed environments from 3D light fields. Densely sampled light fields in the order of 109 light rays allow us to capture the real world in unparalleled detail, but efficiently processing this amount of data to generate an equally detailed reconstruction represents a significant challenge to existing algorithms. We propose an algorithm that leverages coherence in massive light fields by breaking with a number of established practices in image-based reconstruction. Our algorithm first computes reliable depth estimates specifically around object boundaries instead of interior regions, by operating on individual light rays instead of image patches. More homogeneous interior regions are then processed in a fine-to-coarse procedure rather than the standard coarse-to-fine approaches. At no point in our method is any form of global optimization performed. This allows our algorithm to retain precise object contours while still ensuring smooth reconstructions in less detailed areas. While the core reconstruction method handles general unstructured input, we also introduce a sparse representation and a propagation scheme for reliable depth estimates which make our algorithm particularly effective for 3D input, enabling fast and memory efficient processing of "Gigaray light fields" on a standard GPU. We show dense 3D reconstructions of highly detailed scenes, enabling applications such as automatic segmentation and image-based rendering, and provide an extensive evaluation and comparison to existing image-based reconstruction techniques.

Abstract: Points acquired by laser scanners are not intrinsically equipped with normals, which are essential to surface reconstruction and point set rendering using surfels. Normal estimation is notoriously sensitive to noise. Near sharp features, the computation of noise-free normals becomes even more challenging due to the inherent undersampling problem at edge singularities. As a result, common edge-aware consolidation techniques such as bilateral smoothing may still produce erroneous normals near the edges. We propose a resampling approach to process a noisy and possibly outlier-ridden point set in an edge-aware manner. Our key idea is to first resample away from the edges so that reliable normals can be computed at the samples, and then based on reliable data, we progressively resample the point set while approaching the edge singularities. We demonstrate that our Edge-Aware Resampling (EAR) algorithm is capable of producing consolidated point sets with noise-free normals and clean preservation of sharp features. We also show that EAR leads to improved performance of edge-aware reconstruction methods and point set rendering techniques.

Abstract: Modern camera calibration and multiview stereo techniques enable users to smoothly navigate between different views of a scene captured using standard cameras. The underlying automatic 3D reconstruction methods work well for buildings and regular structures but often fail on vegetation, vehicles, and other complex geometry present in everyday urban scenes. Consequently, missing depth information makes Image-Based Rendering (IBR) for such scenes very challenging. Our goal is to provide plausible free-viewpoint navigation for such datasets. To do this, we introduce a new IBR algorithm that is robust to missing or unreliable geometry, providing plausible novel views even in regions quite far from the input camera positions. We first oversegment the input images, creating superpixels of homogeneous color content which often tends to preserve depth discontinuities. We then introduce a depth synthesis approach for poorly reconstructed regions based on a graph structure on the oversegmentation and appropriate traversal of the graph. The superpixels augmented with synthesized depth allow us to define a local shape-preserving warp which compensates for inaccurate depth. Our rendering algorithm blends the warped images, and generates plausible image-based novel views for our challenging target scenes. Our results demonstrate novel view synthesis in real time for multiple challenging scenes with significant depth complexity, providing a convincing immersive navigation experience.

Abstract: We propose a new massively parallel algorithm for constructing high-quality bounding volume hierarchies (BVHs) for ray tracing. The algorithm is based on modifying an existing BVH to improve its quality, and executes in linear time at a rate of almost 40M triangles/sec on NVIDIA GTX Titan. We also propose an improved approach for parallel splitting of triangles prior to tree construction. Averaged over 20 test scenes, the resulting trees offer over 90% of the ray tracing performance of the best offline construction method (SBVH), while previous fast GPU algorithms offer only about 50%. Compared to state-of-the-art, our method offers a significant improvement in the majority of practical workloads that need to construct the BVH for each frame. On the average, it gives the best overall performance when tracing between 7 million and 60 billion rays per frame. This covers most interactive applications, product and architectural design, and even movie rendering.

Abstract: We consider the problem of illumination control in a networked lighting system wherein luminaires have local sensing and actuation capabilities Each luminaire: 1) consists of a light-emitting diode (LED) based light source dimmable by a local controller; 2) is actuated based on sensing information from a presence sensor, that determines occupant presence, and a light sensor, that measures illuminance, within their respective fields of view; and 3) a communication module to exchange control information within a local neighborhood We consider distributed illumination control in such an intelligent lighting system to achieve presence-adaptive and daylight-integrated spatial illumination rendering The rendering is specified as target values at the light sensors, and under these constraints, a local controller has to determine the optimum dimming levels of its associated LED luminaire so that the power consumed in rendering is minimized The formulated optimization problem is a distributed linear programming problem with constraints on exchanging control information within a neighborhood A distributed optimization algorithm is presented to solve this problem and its stability and convergence are studied Sufficient conditions, in terms of parameter selection, under which the algorithm can achieve a feasible solution are provided The performance of the algorithm is evaluated in an indoor office setting in terms of achieved illuminance rendering and power savings

Abstract: Provide dare methods, systems and apparatuses for data brokering between hand held wireless devices (WDs) and data rendering devices (DRDs). DRDs in the form of multimedia devices used for rendering data by printing (e.g. to a networked printer) or displaying video data (e.g., televisions, video monitors, and projectors) are provided with data for rendering at the DRD at the request of WDs. DRDs are capable of receiving data data from a network at the request of a WD and/or directly from a WD as the host and then rendering or displaying the data on devices capable of receiving and processing the data. DRD (e.g. printers and multimedia video devices) can also be controlled by the WD during display of the data and to control display of the data.

Abstract: Spatial judgments are important for many real-world tasks in engineering and scientific visualization. While existing research provides evidence that higher levels of display and interaction fidelity in virtual reality systems offer advantages for spatial understanding, few investigations have focused on small-scale spatial judgments or employed experimental tasks similar to those used in real-world applications. After an earlier study that considered a broad analysis of various spatial understanding tasks, we present the results of a follow-up study focusing on small-scale spatial judgments. In this research, we independently controlled field of regard, stereoscopy, and head-tracked rendering to study their effects on the performance of a task involving precise spatial inspections of complex 3D structures. Measuring time and errors, we asked participants to distinguish between structural gaps and intersections between components of 3D models designed to be similar to real underground cave systems. The overall results suggest that the addition of the higher fidelity system features support performance improvements in making small-scale spatial judgments. Through analyses of the effects of individual system components, the experiment shows that participants made significantly fewer errors with either an increased field of regard or with the addition of head-tracked rendering. The results also indicate that participants performed significantly faster when the system provided the combination of stereo and head-tracked rendering.

Abstract: We present radiance regression functions for fast rendering of global illumination in scenes with dynamic local light sources. A radiance regression function (RRF) represents a non-linear mapping from local and contextual attributes of surface points, such as position, viewing direction, and lighting condition, to their indirect illumination values. The RRF is obtained from precomputed shading samples through regression analysis, which determines a function that best fits the shading data. For a given scene, the shading samples are precomputed by an offline renderer.The key idea behind our approach is to exploit the nonlinear coherence of the indirect illumination data to make the RRF both compact and fast to evaluate. We model the RRF as a multilayer acyclic feed-forward neural network, which provides a close functional approximation of the indirect illumination and can be efficiently evaluated at run time. To effectively model scenes with spatially variant material properties, we utilize an augmented set of attributes as input to the neural network RRF to reduce the amount of inference that the network needs to perform. To handle scenes with greater geometric complexity, we partition the input space of the RRF model and represent the subspaces with separate, smaller RRFs that can be evaluated more rapidly. As a result, the RRF model scales well to increasingly complex scene geometry and material variation. Because of its compactness and ease of evaluation, the RRF model enables real-time rendering with full global illumination effects, including changing caustics and multiple-bounce high-frequency glossy interreflections.

Abstract: Recent years have seen increasing attention and significant progress in many-light rendering, a class of methods for the efficient computation of global illumination. The many-light formulation offers a unified mathematical framework for the problem reducing the full lighting transport simulation to the calculation of the direct illumination from many virtual light sources. These methods are unrivaled in their scalability: they are able to produce artifact-free images in a fraction of a second but also converge to the full solution over time. In this state-of-the-art report, we have three goals: give an easy-to-follow, introductory tutorial of many-light theory; provide a comprehensive, unified survey of the topic with a comparison of the main algorithms; and present a vision to motivate and guide future research. We will cover both the fundamental concepts as well as improvements, extensions, and applications of many-light rendering.

Abstract: Systems and methods are provided for audio device playback failure recovery and redistribution. The systems and methods may be configured to cause a plurality of playback devices to render audio content in a synchronized manner, detect a failure of a first playback device of the plurality of playback devices, determine a playback responsibility of the first playback device, and cause an adjustment of the rendering of the audio content by at least a second playback device of the plurality of playback devices. The systems and methods may further be configured to detect an addition of a new playback device to the plurality of playback devices, determine a playback responsibility of the new playback device for rendering the audio content, and cause an adjustment of the rendering of the audio content by at least a one playback device from the plurality of playback devices.

Abstract: The present invention relates to a method and an audio rendering system comprising at least one portable terminal configured to receive geospatial object data from at least one geospatial object data server. The geospatial object data is interrelated to a geographical position. The at least one portable terminal is further configured to render retrieved geospatial object data into an acoustic scene by a rendering algorithm. The acoustic scene is spatially interrelated to the geographical position. The at least one audio unit is configured to sound a rendered acoustic scene information into at least one ear of a user. The audio rendering system is further configured for rendering retrieved geospatial object data into the acoustic scene based on categorised acoustic scene information representing a corresponding categorised geospatial object data.

Abstract: A computerized system for advising one communicant in electronic communication between two or more communicants has apparatus monitoring and recording interaction between the communicants, software executing from a machine-readable medium and providing analytics, the software functions including rendering speech into text, and analyzing the rendered text for topics, performing communicant verification, and detecting changes in communicant emotion. Advice is offered to the one communicant during the interaction, based on results of the analytics.

Abstract: The CIE colour rendering index (CRI) has been criticized for its poor correlation with the visual colour rendering of many spiked or narrowband sources, its outdated colour space and chromatic adaptation transform and the use of a small number of non-optimal reflectance samples that have enabled lamp manufacturers to tune the spectrum of a light source to yield, in some cases, inappropriately high general CRI values. The CRI2012 metric proposed in this paper addresses these criticisms by combining the most state of the art colorimetric colour difference model, i.e. CAM02-UCS, with a mathematical reflectance set that exhibits a highly uniform spectral sensitivity. A set of 210 real reflectance samples has also been selected to provide additional information on the expected colour shifts when changing illumination.

Abstract: The paper presents some experiments carried out as part of the virtual reconstruction of buildings just documented by partial sketches, or partially built, or no more existing, with the aim (a) to emphasize the use of a semantic construction of the digital model, not only as a means to modeling a building but as a cognitive system, (b) to show conceptual similarity between the treaties and BIM, (c) to propose new and more robust solutions to the 3D modeling from 2D drawings for CH artifacts, able to allow the verification of the assumptions used during the reconstruction pipeline, (d) to make use of interactive technical reference, typically real-time photorealistic rendering, for the visualization of three-dimensional model and of variants snapshots, managed by an iconic for illustrating the method of comparison and guided reading of model’s characters of the steps taken.

Abstract: A system, method, and computer program product are provided for remote rendering of computer graphics. The system includes a graphics application program resident at a remote server. The graphics application is invoked by a user or process located at a client. The invoked graphics application proceeds to issue graphics instructions. The graphics instructions are received by a remote rendering control system. Given that the client and server differ with respect to graphics context and image processing capability, the remote rendering control system modifies the graphics instructions in order to accommodate these differences. The modified graphics instructions are sent to graphics rendering resources, which produce one or more rendered images. Data representing the rendered images is written to one or more frame buffers. The remote rendering control system then reads this image data from the frame buffers. The image data is transmitted to the client for display or processing. In an embodiment of the system, the image data is compressed before being transmitted to the client. In such an embodiment, the steps of rendering, compression, and transmission can be performed asynchronously in a pipelined manner.

Abstract: We present photon beam diffusion, an efficient numerical method for accurately rendering translucent materials. Our approach interprets incident light as a continuous beam of photons inside the material. Numerically integrating diffusion from such extended sources has long been assumed computationally prohibitive, leading to the ubiquitous single-depth dipole approximation and the recent analytic sum-of-Gaussians approach employed by Quantized Diffusion. In this paper, we show that numerical integration of the extended beam is not only feasible, but provides increased speed, flexibility, numerical stability, and ease of implementation, while retaining the benefits of previous approaches. We leverage the improved diffusion model, but propose an efficient and numerically stable Monte Carlo integration scheme that gives equivalent results using only 3--5 samples instead of 20--60 Gaussians as in previous work. Our method can account for finite and multi-layer materials, and additionally supports directional incident effects at surfaces. We also propose a novel diffuse exact single-scattering term which can be integrated in tandem with the multi-scattering approximation. Our numerical approach furthermore allows us to easily correct inaccuracies of the diffusion model and even combine it with more general Monte Carlo rendering algorithms. We provide practical details necessary for efficient implementation, and demonstrate the versatility of our technique by incorporating it on top of several rendering algorithms in both research and production rendering systems.

Abstract: Disclosed is a method for enabling an augmented reality interaction system and a mobile device to overlay a virtual 3D component over a physical 3D component with which the virtual 3D component interacts. The method includes: enabling a user to capture a live camera image of a gaming machine cabinet via a camera on the mobile device; determining if there are image tags on the gaming machine cabinet in the live camera image; producing a virtual rendering of the gaming machine cabinet; determining virtual 3D components to be displayed over an image of a virtual gaming machine cabinet; comparing a virtual depth rendering of the virtual 3D components to a virtual depth rendering of the virtual gaming machine cabinet; and overlaying virtual 3D components without the subtracted elements onto a live camera image of the virtual gaming machine cabinet on a display of the mobile device.

Abstract: Methods and apparatuses for maintaining continuity of augmentations are disclosed. In one embodiment, a method for use with an augmented reality enabled device (ARD) comprises tracking a plurality of objects and a background based at least in part on visual information derived from an image, maintaining states of the plurality of objects based at least in part on information other than the visual information, and providing data for rendering augmentation in response to the states of the plurality of objects.

Abstract: The idea of computer vision as the Bayesian inverse problem to computer graphics has a long history and an appealing elegance, but it has proved difficult to directly implement. Instead, most vision tasks are approached via complex bottom-up processing pipelines. Here we show that it is possible to write short, simple probabilistic graphics programs that define flexible generative models and to automatically invert them to interpret real-world images. Generative probabilistic graphics programs consist of a stochastic scene generator, a renderer based on graphics software, a stochastic likelihood model linking the renderer's output and the data, and latent variables that adjust the fidelity of the renderer and the tolerance of the likelihood model. Representations and algorithms from computer graphics, originally designed to produce high-quality images, are instead used as the deterministic backbone for highly approximate and stochastic generative models. This formulation combines probabilistic programming, computer graphics, and approximate Bayesian computation, and depends only on general-purpose, automatic inference techniques. We describe two applications: reading sequences of degraded and adversarially obscured alphanumeric characters, and inferring 3D road models from vehicle-mounted camera images. Each of the probabilistic graphics programs we present relies on under 20 lines of probabilistic code, and supports accurate, approximately Bayesian inferences about ambiguous real-world images.

Abstract: Free-viewpoint rendering (FVR) has become a popular topic in 3-D research. A promising technology in FVR is to generate virtual views using a single texture image and the corresponding depth image. A critical problem that occurs when generating virtual views is that the regions covered by the foreground objects in the original view may be disoccluded in the synthesized views. In this paper, a depth based disocclusion filling algorithm using patch-based texture synthesis is proposed. In contrast to the existing patch-based virtual view synthesis methods, the filling priority is driven by the robust structure tensor that efficiently reflects the overall structure of an image part and a new confidence term that produces fine synthesis results even near the foreground boundaries. Moreover, the best-matched patch is searched in the background regions and finally it is chosen through a new patch distance measure. Significant superiority of the proposed method over the state-of-the-art methods is presented by comparing the experimental results.

Abstract: Virtual try-on applications have become popular because they allow users to watch themselves wearing different clothes without the effort of changing them physically. This helps users to make quick buying decisions and, thus, improves the sales efficiency of retailers. Previous solutions usually involve motion capture, 3D reconstruction or modeling, which are time consuming and not robust for all body poses. Our method avoids these steps by combining image-based renderings of the user and previously recorded garments. It transfers the appearance of a garment recorded from one user to another by matching input and recorded frames, image-based visual hull rendering, and online registration methods. Using images of real garments allows for a realistic rendering quality with high performance. It is suitable for a wide range of clothes and complex appearances, allows arbitrary viewing angles, and requires only little manual input. Our system is particularly useful for virtual try-on applications as well as interactive games.

Abstract: We propose a new method for occlusion culling in the computation of a hologram based on the mutual conversion between light-rays and wavefront. Since the occlusion culling is performed with light-ray information, conventional rendering techniques such as ray-tracing or image-based rendering can be employed. On the other hand, the wavefront is derived for the calculation of light propagation, the hologram of 3-D objects can be obtained in high accuracy. In the numerical experiment, we demonstrate that our approach can reproduce a high-resolution image for deep 3-D scene with correct occlusion effect between plural objects.

Abstract: BACKGROUND AND PURPOSE: CT in low dose technique is the criterion standard imaging modality for evaluation of the paranasal sinus. Our aim was to evaluate the dose-reduction potential of a recently available sinogram-affirmed iterative reconstruction technique, regarding noise, image quality, and time duration when evaluating this region. MATERIALS AND METHODS: CT was performed on a phantom head at different tube voltages (120 kV, 100 kV) and currents (100 mAs, 50 mAs, 25 mAs). Each protocol was reconstructed (in soft tissue and bony kernel) by using standard filtered back-projection and 5 different SAFIRE strengths, and image noise was evaluated. Subjective image quality was evaluated on noise-aligned image triplets acquired at tube currents of 100% (FBP), 50% (SAFIRE), and 25% (SAFIRE) by using a 5-point scale (1 = worst, 5 = best). The time duration for image reconstruction was noted for calculations with FBP and SAFIRE. RESULTS: SAFIRE reduced image noise by 15%–85%, depending on the iterative strength, rendering kernel, and dose parameters. Noise reduction was stronger at a bone kernel algorithm both in 1- and 3-mm images (P CONCLUSIONS: For CT of the paranasal sinus, SAFIRE algorithms are suitable for image-noise reduction. Because image quality decreases with dosage, careful choice of the appropriate iterative method is necessary to achieve an optimal balance between image noise and quality.

Abstract: Systems and methods are provided for inserting advertisements into a video. In accordance with one implementation, a method is provided that includes processing a data stream including a video content selection to extract a metatag that models a template object appearing within a first content segment featuring primary content of the video content selection, the template object enabling a transformation of an advertising object. Further, the method includes identifying an advertising descriptor based on characteristics of a scene and the metatag extracted from the first content segment and selecting the advertising object based on the identified advertising descriptor. The method also includes modifying the video content selection by inserting the advertising object in the data stream such that the advertising object is integrated into a scene of the video content selection and transmitting the video content selection to a recipient.

Abstract: The invention provides an augmented-reality-based three-dimensional interactive learning system and an augmented-reality-based three-dimensional interactive learning method. The system comprises an information processing device, a photographic device, a display device and at least one physical teaching aid, wherein the physical teaching aid is provided with identification information; the photographic device is used for performing video acquisition on a real environment after augmented reality application is started; and the information processing device comprises an identification module for identifying the identification information on the physical teaching aid, an orientation calculation module for calculating the spatial orientation information of the physical teaching aid, and a three-dimensional rendering module for acquiring a three-dimensional model corresponding to the identification information, rendering and generating a corresponding virtual object, and placing the virtual object at a corresponding position in a video image for display according to the spatial orientation information of the physical teaching aid. According to the system and the method, the real environment and the virtual object are overlapped in the same scenario in real time, so that a more vivid sensory experience is provided for a user, and meanwhile, a teaching effect is improved by utilizing the instinct of a person for the recognition of a three-dimensional space.

Abstract: In this paper, we present a GPU-based out-of-core rendering approach under the many-lights rendering framework. Many-lights rendering is an efficient and scalable rendering framework for a large number of lights. But when the data sizes of lights and geometry are both beyond the in-core memory storage size, the data management of these two out-of-core data becomes critical and challenging. In our approach, we formulate such a data management as a graph traversal optimization problem that first builds out-of-core lights and geometry data into a graph, and then guides shading computations by finding a shortest path to visit all vertices in the graph. Based on the proposed data management, we develop a GPU-based out-of-GPU-core rendering algorithm that manages data between the CPU host memory and the GPU device memory. Two main steps are taken in the algorithm: the out-of-core data preparation to pack data into optimal data layouts for the many-lights rendering, and the out-of-core shading using graph-based data management. We demonstrate our algorithm on scenes with out-of-core detailed geometry and out-of-core lights. Results show that our approach generates complex global illumination effects with increased data access coherence and has one order of magnitude performance gain over the CPU-based approach.