Gaussian splatting is a real-time 3D scene representation that models a scene as millions of small 3D Gaussian primitives, each with a position, covariance, colour, and opacity - enabling photorealistic rendering at interactive frame rates without neural network inference.
Introduced as 3D Gaussian Splatting (3DGS), the technique reconstructs a scene from multi-view images by optimising a set of anisotropic 3D Gaussians. Each Gaussian is defined by its mean (position), covariance matrix (shape and orientation), opacity, and spherical harmonics coefficients (view-dependent colour).
Rendering works by projecting ("splatting") these Gaussians onto the image plane and alpha-compositing them front-to-back. Because there is no per-pixel ray marching, rendering is orders of magnitude faster than Neural Radiance Fields (NeRFs) while achieving comparable or better visual quality.
Variants like WildGaussians handle in-the-wild captures with appearance variation, and Gaussian Frosting adds a thin shell of Gaussians on top of a mesh for hybrid mesh+splat rendering. The format is also increasingly used as an intermediate representation in 3D generative pipelines.
Datameister uses Gaussian splatting in scan-to-3D pipelines - converting point clouds and images into photorealistic reconstructions for inspection, digital twins, and media production.
Datameister's presence at ECCV 2024 in Milan was a deep dive into the latest advancements in Computer Vision. This blogpost explores groundbreaking techniques like 3D Gaussian Splatting and its enhancements, such as WildGaussians and Gaussian Frosting, showcasing their potential to revolutionize real-time rendering and scene representation.
Discover the exciting advancements in image-based rendering driven by AI. From light fields to SMoE, Gaussian splatting, and NeRFs, these breakthrough techniques have revolutionized the field. Explore the mathematical models of light and practical challenges in capturing light fields. Uncover how deep learning and diffusion models have overcome longstanding issues. At Datameister, we're actively following these developments and exploring ways to contribute further. Stay tuned for more!
