Abstract: The accelerated collection of detailed real-world 3D data in the form of ever-larger point clouds is sparking a demand for novel visualization techniques that are capable of rendering billions of point primitives in real-time. We propose a software rasterization pipeline for point clouds that is capable of rendering up to two billion points in real-time (60 FPS) on commodity hardware. Improvements over the state of the art are achieved by batching points, enabling a number of batch-level optimizations before rasterizing them within the same rendering pass. These optimizations include frustum culling, level-of-detail (LOD) rendering, and choosing the appropriate coordinate precision for a given batch of points directly within a compute workgroup. Adaptive coordinate precision, in conjunction with visibility buffers, reduces the required data for the majority of points to just four bytes, making our approach several times faster than the bandwidth-limited state of the art. Furthermore, support for LOD rendering makes our software rasterization approach suitable for rendering arbitrarily large point clouds, and to meet the elevated performance demands of virtual reality applications.

Abstract: With the speedy increase of display resolution and the demand for interactive frame rate, rendering acceleration is becoming more critical for a wide range of virtual reality applications. Foveated rendering addresses this challenge by rendering with a non-uniform resolution for the display. Motivated by the non-linear optical lens equation, we present rectangular mapping-based foveated rendering (RMFR), a simple yet effective implementation of foveated rendering framework. RMFR supports varying level of foveation according to the eccentricity and the scene complexity. Compared with traditional foveated rendering methods, rectangular mapping-based foveated rendering provides a superior level of perceived visual quality while consuming minimal rendering cost.

Abstract: ANARI is a new 3-D rendering API, an emerging Khronos standard that enables visualization applications to leverage the state-of-the-art rendering techniques across diverse hardware platforms and rendering engines. Visualization applications have historically embedded custom-written renderers to enable them to provide the necessary combination of features, performance, and visual fidelity required by their users. As computing power, rendering algorithms, dedicated rendering hardware acceleration operations, and associated low-level APIs have advanced, the effort and costs associated with maintaining renderers within visualization applications have risen dramatically. The rising cost and complexity associated with renderer development creates an undesirable barrier for visualization applications to be able to fully benefit from the latest rendering methods and hardware. ANARI directly addresses these challenges by providing a high-level, visualization-oriented API that abstracts low-level rendering algorithms and hardware acceleration details while providing easy and efficient access to diverse ANARI implementations, thereby enabling visualization applications to support the state-of-the-art rendering capabilities.

Abstract: The advancements in hardware have drawn more attention than ever to high-quality offline rendering with modern stream processors, both in the industry and in research fields. However, the graphics APIs are fragmented and existing shading languages lack high-level constructs such as polymorphism, which adds complexity to developing and maintaining cross-platform high-performance renderers. We present LuisaRender1, a high-performance rendering framework for modern stream-architecture hardware. Our main contribution is an expressive C++-embedded DSL for kernel programming with JIT code generation and compilation. We also implement a unified runtime layer with resource wrappers and an optimized Monte Carlo renderer. Experiments on test scenes show that LuisaRender achieves much higher performance than existing research renderers on modern graphics hardware, e.g., 5--11× faster than PBRT-v4 and 4--16× faster than Mitsuba 3.

Abstract: Real-time rendering techniques, developed for computer games, offer great opportunities in Virtual Production. Ray Tracing has been used for CGI movies for many years but it is only recently that its application in real-time has become practical. This is partly due to improved algorithms but mostly advanced hardware such as the Nvidia Geforce RTX 3000 series of cards which provide hardware support for real-time lighting thus improving the quality of the rendered images in CGI. We conducted a series of tests for rendering of a Virtual Production scene in Unreal game engine. Images are rendered in 4K and output to a network distribution system where the image is broken down into a series of smaller images each rendered onto LED screens. Results were plotted to show the comparison of render times between two graphics workstations using Nvidia RTX A6000 GPU cards and Nvidia RTX A3090 GPU. Our findings state that whilst RTX produces better image quality the gains might not be worth the additional hardware cost required by the high-end graphic cards. It might also be optimal to split the rendering of the scene across multiple computers.

Abstract: Virtual Reality (VR) is now a well-established technology which offers realistic and immersive virtual worlds to the user usually by means of Head-Mounted Displays (HMDs). Actually, these devices can also be backed by a cloud server which can host the game server or even directly render the virtual world, as in the well-known cloud gaming paradigm. However, due to the drastically low latencies that technology requires, it is more convenient, when possible, to use servers that are as close to the users as possible. As a consequence, implementing the game or the render server in the Fog Computing layer is a concrete possibility. A research investigation has been carried out by using the Meta Quest 2 device, which are the QoE trade-offs, in terms of graphic quality and network performance, both in the case in which the HMD performs the 3d rendering locally by using a sample game written with Unreal Engine and in the case in which the 3d rendering is done in the Fog by means of the nVidia CloudXR framework and Oculus Air Link. The results of the proposed experiments reveal that the remote rendering offers a stable frame rate against a higher quality image. Instead, local rendering sets the best possible graphics quality against the optimal frame rate. Additionally, the utilization of remote rendering to perform video compression in the case of decreasing the available bandwidth to adjust graphics quality and FPS has also been analyzed. The same does not hold for Motion-to-Photon latency, which increases with distance, reducing the general QoE.

Abstract: One of the main uses for OpenCL is (scientific) compute applications where graphical rendering is done externally, after the simulation has finished. However separating simulation and rendering has many disadvantages, especially the extreme slowdown caused by copying simulation data from device to host, and needing to store raw data on the hard drive, taking up hundreds of gigabyte, just to visualize preliminary results. A much faster approach is to implement both simulation and rendering in OpenCL. The rendering kernels have direct read-only access to the raw simulation data that resides in ultra-fast GPU memory. This eliminates all PCIe data transfer but camera parameters and finished frames, allowing for interactive visualization of simulation results in real time while the simulation is running. This is an invaluable tool for rapid prototyping. Although OpenCL does not have existing functionality for graphical rendering, being a general compute language, it allows for implementing an entire graphics engine, such that no data has to be moved to the CPU during rendering. On top, specific low-level optimizations make this OpenCL graphics engine outperform any existing rendering solution for this scenario, enabling drawing billions of lines per second and fluid raytracing in real time on even non-RTX GPUs. This combination of simulation and rendering in OpenCL is demonstrated with the software FluidX3D [3] - a lattice Boltzmann method (LBM) fluid dynamics solver. The first part will briefly introduce the numerical method for simulating fluid flow in a physically accurate manner. After introducing the LBM, the optimizations to make it run at peak efficiency are discussed: Being a memory-bound algorithm, coalesced memory access is key. This is achieved through array-of-structures data layout as well as the one-step-pull scheme, a certain variant of the LBM streaming step. One-step-pull leverages the fact that the misaligned read penalty is much smaller than the misaligned write penalty on almost all GPUs. Roofline analysis shows that with these optimizations, the LBM runs at 100% efficiency on the fastest data-center and gaming GPUs [5]. To simulate free surface flows, the LBM is extended with the Volume-of-Fluid (VoF) model. An efficient algorithm has been designed to vastly accelerate the challenging surface tension computation [4]. This extremely efficient VoF-LBM GPU implementation allows covering new grounds in science: FluidX3D has been used to simulate more than 1600 raindrop impacts to statistically evaluate how microplastics transition from the ocean surface into the atmosphere when the spray droplets are generated during drop impact [6]. At the same power consumption, with existing CPU-parallelized codes, compute time would have been several years, whilst with FluidX3D it was about a week. The second part will focus on real time rendering with OpenCL, especially raytracing. Rasterization on the GPU is parallelized not over pixels but lines/triangles instead, making runtime mostly independent of screen resolution and lightning fast. Each line/triangle is transformed with the camera parameters from 3D to 2D screen coordinates and then rasterized onto the frame (integer array) with Bresenham algorithm [2] and z-buffer. The raytracing graphics are based on a combination of fast ray-grid traversal and marching-cubes, leveraging that the computational grid from the LBM already is an ideal acceleration structure for raytracing. The idea of raytracing is simple: Through each pixel on the screen, shoot a reverse light ray out of the camera and see where it intersects with a surface in the scene. Then (recursively) calculate reflected/refracted rays and mix the colors. If a ray doesn’t intersect with anything, its color is determined by the skybox image via UV mapping and bilinear pixel interpolation. With mesh surfaces consisting of many triangles, computation time quickly becomes a problem, as for each ray all triangles have to be tested for intersection. To overcome this, an acceleration structure is required. While computer games often use a bounding volume hierarchy, the LBM already provides an ideal alternative acceleration structure: the simulation grid. The corresponding algorithm is called ray-grid traversal: When a ray shoots through the 3D grid, intersections with the surface only have to be checked for at each traversed grid cell rather than the entire grid. In each traversed grid cell, the 0-5 surface triangles are generated on-the-fly with the marching-cubes algorithm and ray-triangle intersections are checked with the Möller-Trumbore algorithm. If an intersection has been found, only afterwards the normals are calculated on the 8 grid points spanning the cell, and are trilinearly interpolated to the intersection coordinates. The so interpolated surface normal makes the raytraced surface appear perfectly smooth. On the GPU, the ray(s) for each pixel on screen are computed in parallel, vastly speeding up rendering. It is of key importance how to align the OpenCL workgroups on the 2D array of screen pixels: best performance is achieved for 8x8 pixel tiles; this is about 50% faster than 64x1 tiles, because with small, square-ish tiles, all rays of the workgroup are more likely to traverse the same grid cells, greatly improving memory broadcasting. In ray-grid traversal, 8 isovalues spanning a cell have to be loaded from GPU memory for each traversed cell. Once the triangle intersection has been found, the gradient on each of the 8 cell isovalues is calculated with central differences. Instead of loading an additional 6 isovalues for each of the 8 grid points, their isovalues are reused such that only 24 additional isovalues are loaded. For marching-cubes, the algorithm by Paul Bourke [1] is implemented in OpenCL. With 16-/8-bit integers, bit-packing and symmetry, the tables are reduced to 1/8 of their original size and stored in constant memory space. For computing the cube index, branching is eliminated by bit operations. The Möller-Trumbore algorithm [7] is implemented in an entirely branchless manner. This raytracing implementation is fast enough to run in real time for even the largest lattice dimensions that fit into the memory of a GPU. Finally, the combined VoF-LBM simulation and raytracing implementation is demonstrated on the most realistic simulation of an impacting raindrop ever done [8].

Abstract: In the development of space mission simulation system, the fidelity of real-time scene rendering is one of the key indexes of simulation system. This paper presents a new shader technology based on physical rendering. The technique was validated by rendering tests in the engine. The simulation results show that the method can effectively improve the fidelity of scene rendering and provide technical support for the development of high-fidelity space mission simulation system.

Abstract: Real‐time rendering of visual scenes is a key requirement in many emerging fields like augmented and virtual reality (AR/VR), computer graphics, and so forth. Latency in rendering the scenes due to massive shading calculations and complete rasterization of view windows is undesirable in real‐time AR/VR scenarios. Real‐time determination of occluded and visible objects is one of the significant problems the rendering process faces. The classical Z‐buffer algorithm used for this purpose requires a lot of memory to store depth values, hence maintaining a buffer. Due to scan‐line tendency, its time complexity is also too high. This article presents a new perspective on a fundamental Z‐buffer algorithm. The proposed approach is not entirely a scan‐line method. It is based on divide and conquer strategy, and it also works on the concept of the front‐to‐back concept, where rays are emitted from a viewpoint toward the objects. Based on the proposed algorithm, partially occluded game objects can also be efficiently determined. Only partial shading calculations will be done using the proposed rendering factor calculation to render those objects. It will also speed up the real‐time processing without using any GPU. This way, the proposed approach reduces the time and space complexity of the rendering process.

Abstract: Real-Time rendering applications surrounds us in our everyday life, and this area of software just keep growing with newer systems, devices, and technologies. Starting from a large desktop PC down to small, even embedded devices, almost every computer device contains a graphics processor. These graphics units are programmable using Graphics APIs, and usually we use these libraries from C or C++ thanks to their low-level capabilities and because of the need for high performance. We want to present the starting steps of replacing the C or C++ language with .NET C# for developing multi-platform real-time graphical applications. Using the modern .NET environment, we can use Graphics APIs for rendering onto common .NET UI Frameworks while consuming all our previously implemented C# libraries and .NET technologies in the same application. To maintain compatibility with multiple platforms we are developing a library system to be able to use different Graphics APIs from the same C# source-code. In this paper, we are proposing some methods and considerations for implementing a library to be able to use the Vulkan and OpenGL APIs through a single C# codebase. We provide solutions for multi-platform rendering onto UI and dealing with the low-level challenges of using the two deeply different APIs to be able to deliver our unique real-time graphics into C# applications.

Abstract: This study aims to formulate a concept for an online rendering website based on renderers’ experience on using rendering software (V-Ray, Enscape and Lumion). Rendering software is essential to professionals (architects and 3D visualisers) and students alike. We conduct a survey of students, architects, and 3D visualisers regarding their rendering experience. There are various complaints about rendering: lengthy, costly hardware parts, and hardware incompatibility. We see a solution in providing an online-based rendering service on a website by implementing cloud computing technology. Accordingly, further research needs to uncover other possible solutions to provide the ultimate rendering experience.

Abstract: Advances in entertainment-targeted rendering technology have been leveraged for scientific analysis. Recent progress in both hardware and software capabilities have spurred development in analytic rendering: rendering capabilities optimized for analysis, particularly 3-D spatial analysis. These efforts also leverage hardware-accelerated ray tracing for high-fidelity rendering, which unlocks the potential to use such acceleration methods directly in ray-based simulations such as radiative transfer. This special issue presents the new ANARI API standard for analytic rendering and examples of analytic rendering and hardware-accelerated simulation where such APIs could be used.

Abstract: Graphics rendering remains one of the most compute-intensive and memory-bound applications of GPUs and has been driving their push for performance and energy efficiency since its inception. Early GPU architectures focused only on accelerating graphics rendering and implemented dedicated a fixed-function rendering units. Today’s GPUs have become more programmable to address the complexity and diversity of modern graphics workloads while still accelerating several components of the graphics pipeline in fixed-function hardware.Generalizing the GPU microarchitecture and implement some of its graphics hardware blocks in software can save area that can be used to expand the generic pipeline, especially in mobile systems-on-chips environments where power and area is scarce.In this work, we propose a RISC-V-based hybrid GPU architecture that accelerates the graphics pipeline without paying the cost of a full hardware graphics pipeline. We evaluated the design on an Altera Arria 10 FPGA running at 200 MHz.

Abstract: In the present article, performance of two computer graphics application programming interfaces (APIs), OpenGL and Vulkan is compared using the application developed with C++ programming language. One of the two API’s for scene rendering can be chosen before compilation of program. OpenGL Shading Language (GLSL) is used to create shader for OpenGL backend. Simple shader with implementation of Phong-style shading was created in GLSL then compiled to SPIR-V language for Vulkan support. Results indicate that usage of Vulkan may double the framerate compared to OpenGL. Performance boost was achieved using pre-recorded command buffers, minimization of descriptor set count, using one vertex buffer and one index buffer per scene and using mip- maps for rendering. Framerate was captured using RenderDoc software.

Abstract: The paper describes how MIP-mapping and paging can be used to represent not only terrain imagery, but also terrain elevation. Previously the only things missing to implement Earth coverage were computing power, input/output bandwidth, graphics processing units (GPUs) and techniques to deal with large data sets. The article describes the rendering method that uses the GPU for most of the calculations. Modern graphics accelerators for personal computers allow you to solve problems that require the generation of images of the visual environment of photographic quality in real time. High-quality visualization of large continuous spaces of 3D vegetation is one of the requirements when creating most land transport simulators, as well as many virtual reality applications. Representation of large forest spaces in the form of free-standing tree models that have an acceptable visual quality for close-up display from the observer, it is impossible due to the huge number of primitives required for visualization. The article proposes use parallel calculations in GPU to accelerate rendering. We successfully integrated proposed visualization method into the standard rendering pipeline. For considered tests the application with GPU average ten times faster, than the version using only CPU. The proposed method takes into account the features of the architecture of modern GPUs and allows you to distribute the load on the display between the CPU and GPU. In this case, the method does not require significant resources for visualization. The implementation of the method showed that received speed in most cases is sufficient for the effective application of the method in computer games. The paper also proposes a method for high-quality visualization of large continuous spaces of 3D vegetation.

Abstract: Recently, virtual reality (VR) technology has been widely used in medical, military, manufacturing, entertainment, and other fields. These applications must simulate different complex material surfaces, various dynamic objects, and complex physical phenomena, increasing the complexity of VR scenes. Current computing devices cannot efficiently render these complex scenes in real time, and delayed rendering makes the content observed by the user inconsistent with the user's interaction, causing discomfort. Foveated rendering is a promising technique that can accelerate rendering. It takes advantage of human eyes' inherent features and renders different regions with different qualities without sacrificing perceived visual quality. Foveated rendering research has a history of 31 years and is mainly focused on solving the following three problems. The first is to apply perceptual models of the human visual system into foveated rendering. The second is to render the image with different qualities according to foveation principles. The third is to integrate foveated rendering into existing rendering paradigms to improve rendering performance. In this survey, we review foveated rendering research from 1990 to 2021. We first revisit the visual perceptual models related to foveated rendering. Subsequently, we propose a new foveated rendering taxonomy and then classify and review the research on this basis. Finally, we discuss potential opportunities and open questions in the foveated rendering field. We anticipate that this survey will provide new researchers with a high-level overview of the state of the art in this field, furnish experts with up-to-date information and offer ideas alongside a framework to VR display software and hardware designers and engineers.

Abstract: Volume data is found in many important scientific and engineering applications. Rendering this data for visualization at high quality and interactive rates for demanding applications such as virtual reality is still not easily achievable even using professional-grade hardware. We introduce FoVolNet -- a method to significantly increase the performance of volume data visualization. We develop a cost-effective foveated rendering pipeline that sparsely samples a volume around a focal point and reconstructs the full-frame using a deep neural network. Foveated rendering is a technique that prioritizes rendering computations around the user's focal point. This approach leverages properties of the human visual system, thereby saving computational resources when rendering data in the periphery of the user's field of vision. Our reconstruction network combines direct and kernel prediction methods to produce fast, stable, and perceptually convincing output. With a slim design and the use of quantization, our method outperforms state-of-the-art neural reconstruction techniques in both end-to-end frame times and visual quality. We conduct extensive evaluations of the system's rendering performance, inference speed, and perceptual properties, and we provide comparisons to competing neural image reconstruction techniques. Our test results show that FoVolNet consistently achieves significant time saving over conventional rendering while preserving perceptual quality.

Abstract: Rendering participating media using photon mapping has been a hot subject in computer graphics, but the rendering process generally needs much time cost. In this paper, we propose an octree-based photon storage structure and a corresponding photon query algorithm to solve this problem, which can be used to reduce the time for rendering homogeneous participating media. Our algorithm is based on the traditional photon mapping. Firstly, the photons in the volume medium are recursively divided according to their spatial positions, and the divided data are stored by bounding boxes. All bounding boxes are organized to generate an octree structure, and finally a cluster of photons is stored in the leaf nodes of the octree. Using the octree structure to store photon information can reduce the complexity of the data structure. At the same time, the spatial localization of arbitrary sampling points in the ray can be performed to quickly find the group of photons contributing to these points, because of the octree structure during the collection process. Therefore, our algorithm reduces the time cost of photon queries and finally improves the efficiency of the photon mapping algorithm for rendering the participating media.

Abstract: With the healthy and rapid development of the new generation of information technology, the computer field has achieved a great step of development, and at the same time, there have been three technological development trends in this field: one is human-machine interface naturalization, two is data processing and scientific calculation visualization, and three is computer simulation graphics. The above three technologies are based on computer graphics technology, among which real-time 3D graphics technology and software system is the key technology of computer graphics technology. Dynamic and challenging, 3D graphics is a feature of this state-of-the-art technology, which can be used in a wide range of industries, such as clothing design, virtual surgical systems, urban planning, social networking and so on. Real-time 3D graphics technology has become an indispensable part of software technology. Virtual reality is to create a virtual scene, and its key technology is three-dimensional graphics technology, which also uses real-time simulation and multidimensional development tools. With the improvement of hardware level, as well as the development of many graphics algorithms and core technologies, the real-time 3D rendering effect has been closer to the real physical phenomenon. On the basis of the three-dimensional landscape, in view of the graphics rendering has carried on the detailed design, focus on improved lighting and rendering graphics rendering technology processing, in order to achieve the desired effect, first of all the virtual engine overall structure has carried on the detailed design, and USES the rendering light processing technology and image rendering technology to apply colours to a drawing processing, And finally designed a virtual engine system based on three-dimensional landscape roaming, proposed the application of graphics and image rendering technology research, to achieve the expected results, compared with the previous, efficiency has been greatly improved. However, the complexity of algorithms and many specialized core technologies have made it much more difficult to develop a realistic and efficient real-time 3D application. The virtual engine uses full access to the programmable rendering pipeline to process real-time graphics. The engine uses a delayed coloring rendering architecture. This paper will focus on this core rendering architecture in detail and design a 3D virtual engine based immersion display system design.

Abstract: The rapid rendering of complex 3D scene has always been one of the challenging problems in computer graphics. At present, the most commonly used rendering architecture is the method based on BS architecture. The smooth display method of complex 3D scene based on B / S architecture not only needs to solve the problem of network transmission, but also needs to consider the load caused by complex 3D scene on the server. Because the displayed 3D scene often contains a lot of geometric information and high fidelity texture information, this paper will analyze the characteristics of this kind of scene, and then propose a rendering acceleration method of complex scene based on parallel processing. Parallel processing is adopted. On the other hand, it is also convenient to apply cloud computing technology to the smooth display of complex 3D scenes based on B / S architecture.

Abstract: In computer graphics, opaque and semitransparent objects are handled very differently from one another. Rendering opaque objects is relatively easy, because every time we attempt to draw a pixel, we either fully replace the existing color value (if the depth test passes) or do not modify the existing value (if the depth test fails). With transparent rendering, the color of the pixel being rendered is blended with the existing value in the color buffer. When multiple transparent objects overlap on the same pixel, the order in which we render them matters, so we need to sort all transparent objects prior to rendering. Rendering transparent objects is more computationally intensive than rendering opaques, and in this chapter, we will see why.

Abstract: Many visually complex features related to graphics and rendering will be discussed in this chapter. We will also cover the particle and the animation systems in FXGL in detail. In particular, we will consider the rendering pipeline in FXGL, the UI layer, as well as the particle and animation systems.

Abstract: Neural Radiance Fields (NeRF) have demonstrated superior novel view synthesis performance but are slow at rendering. To speed up the volume rendering process, many acceleration methods have been proposed at the cost of large memory consumption. To push the frontier of the efficiency-memory trade-off, we explore a new perspective to accelerate NeRF rendering, leveraging a key fact that the viewpoint change is usually smooth and continuous in interactive viewpoint control. This allows us to leverage the information of preceding viewpoints to reduce the number of rendered pixels as well as the number of sampled points along the ray of the remaining pixels. In our pipeline, a low-resolution feature map is rendered first by volume rendering, then a lightweight 2D neural renderer is applied to generate the output image at target resolution leveraging the features of preceding and current frames. We show that the proposed method can achieve competitive rendering quality while reducing the rendering time with little memory overhead, enabling 30FPS at 1080P image resolution with a low memory footprint.

Abstract: We propose a software rasterization pipeline for point clouds that is capable of brute-force rendering up to two billion points in real time (60fps). Improvements over the state of the art are achieved by batching points in a way that a number of batch-level optimizations can be computed before rasterizing the points within the same rendering pass. These optimizations include frustum culling, level-of-detail rendering, and choosing the appropriate coordinate precision for a given batch of points directly within a compute workgroup. Adaptive coordinate precision, in conjunction with visibility buffers, reduces the number of loaded bytes for the majority of points down to 4, thus making our approach several times faster than the bandwidth-limited state of the art. Furthermore, support for LOD rendering makes our software-rasterization approach suitable for rendering arbitrarily large point clouds, and to meet the increased performance demands of virtual reality rendering.