Table of Contents

1. Introduction
2. Real-Time GPU-Accelerated Gaussian Splatting with NVIDIA DesignWorks Sample
3. Techniques Demonstrated in vk_gaussian_splatting
   • Rendering splats with mesh shaders
   • Rendering splats with vertex shaders
   • GPU-based Radix Sort for back-to-front sorting
   • CPU-based asynchronous sorting strategy
4. Features of vk_gaussian_splatting Sample
   • Visualization modes for inspecting Gaussian splats
   • Benchmarking system for real-time profiling
   • Memory consumption details
   • GPU timings for different techniques
   • Graphical reports for analysis
5. Conclusion
6. References

Introduction

The introduction provides an overview of the vk_gaussian_splatting sample project, which demonstrates real-time Gaussian splatting using the Vulkan API. Gaussian splatting is a volume rendering technique that efficiently represents radiance fields, making it ideal for rendering photorealistic scenes in real-time applications.

Real-Time GPU-Accelerated Gaussian Splatting with NVIDIA DesignWorks Sample

This section discusses the NVIDIA DesignWorks sample project vk_gaussian_splatting, which serves as a testbed for exploring and comparing various approaches to real-time visualization of 3D Gaussian splatting. The sample leverages Vulkan to demonstrate rendering techniques for Gaussian splats, emphasizing performance, quality, and implementation trade-offs.

Techniques Demonstrated in vk_gaussian_splatting

• Rendering splats with mesh shaders: One of the approaches showcased in the sample involves using mesh shaders to render Gaussian splats efficiently.
• Rendering splats with vertex shaders: Another technique demonstrated is rendering splats using vertex shaders, providing alternative methods for visualization.
• GPU-based Radix Sort for back-to-front sorting: The sample includes a GPU-based Radix Sort implemented in a compute pipeline to ensure correct back-to-front sorting for alpha compositing.
• CPU-based asynchronous sorting strategy: An asynchronous sorting strategy is implemented using the C++ STL multithreaded sort function for efficient sorting of Gaussian splats.

Features of vk_gaussian_splatting Sample

• Visualization modes for inspecting Gaussian splats: Users can explore various visualization modes to inspect different aspects of Gaussian splats, including spherical harmonics, splats, and point density.
• Benchmarking system for real-time profiling: The sample provides a benchmarking system for real-time profiling, enabling developers to analyze performance metrics during rendering.
• Memory consumption details: Detailed information on RAM and VRAM memory consumption is available to understand data streaming for rendering purposes.
• GPU timings for different techniques: GPU timings for each stage of the rendering techniques are provided to identify workload distribution and potential bottlenecks.
• Graphical reports for analysis: Graphical reports are generated based on performance numbers to facilitate in-depth analysis of rendering processes.

Conclusion

The vk_gaussian_splatting sample serves as a valuable resource for developers interested in experimenting with Gaussian splatting rendering techniques and Vulkan optimizations. It offers a starting point for exploring real-time rendering of neural radiance fields, providing insights into performance optimization and quality enhancement strategies.

References

• GitHub Repository: nvpro-samples/vk_gaussian_splatting