tfg.rendering.splat.splat_at_pixel_centers

Splat a buffer of XYZ, RGBA samples onto a pixel grid of the same size.

tfg.rendering.splat.splat_at_pixel_centers(
    xyz_rgba: Tuple[tf.Tensor, tf.Tensor]
) -> Tuple[tf.Tensor, tf.Tensor, tf.Tensor]

This is a specialized splatting function that takes a multi-layer buffer of screen-space XYZ positions and RGBA colors and splats each sample into a buffer of the same size, using a 3x3 Gaussian kernel of variance 0.25. The accumulated layers are then composited back-to-front.

The specialized part is that the 3x3 kernel is always centered on the pixel-coordinates of the sample in the input buffer, not the XY position stored at that sample, but the weights are defined by using the XY position. Computing weights w.r.t. the XY positions, rather than the pixel-centers, allows gradients to flow from the output RGBA back to the XY positions. When used in rasterize-then-splat, XY positions will always coincide with the pixel centers, so the forward computation is the same as if the XY positions defined the position of the splat.

When splatting, the Z of the splat is compared with the Z of the layers under the splat sample. The sample is accumulated into the layer with the Z closest to the Z of the splat itself.

xyz_rgba a tuple of a float32 tensor of rasterized XYZ positions with shape [num_layers, height, width, 3] and a tensor of RGBA colors [num_layers, height, width, 4]. Passed as a tuple to support tf.vectorized_map.

A tensor of shape [height, width, 4] with RGBA values, as well as [num_layers, height, width, 4] tensor of accumulated and normalized colors for visualization and debugging.