Help Needed: Converting PCD to Gaussian Splatting PLY

[masterAI359]

Body

I’m using FastAPI to convert PCD files (sent from the frontend) into Gaussian Splatting PLY files.
However, the converted PLY files cannot be opened in platforms such as:

• https://sandbox.babylonjs.com/
• https://playcanvas.com/
• https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php

When I try to import them, the browser crashes.

Has anyone encountered this issue or could guide how to convert PCD to PLY for Gaussian Splatting properly?
Any help would be greatly appreciated.

Guidelines

• I have read and understood this category's guidelines before making this post.

[26nikhilkumar]

A Gaussian Splatting PLY is not a regular point cloud or mesh PLY. It must contain a very specific set of vertex properties, stored as binary little endian, and several of those properties are encoded, not raw. If you export a “plain” PLY from PCD with only x y z r g b, web viewers that expect Gaussian fields try to parse missing attributes and either throw or run out of bounds. The commonly used schema looks like this (header excerpt shown in the official repo’s issues):

ply
format binary_little_endian 1.0
element vertex N
property float x
property float y
property float z
property float nx
property float ny
property float nz
property float f_dc_0
property float f_dc_1
property float f_dc_2
property float f_rest_0
...
property float f_rest_44
property float opacity
property float scale_0
property float scale_1
property float scale_2
property float rot_0
property float rot_1
property float rot_2
property float rot_3
end_header

That layout and naming is what the popular viewers expect. If your file lacks these fields, or if the types or order differ, you will see crashes or garbage. ([GitHub]1)

On top of that, several fields are stored in “training space” and must be decoded by the viewer. In particular:

• f_dc_0..2 are spherical harmonics DC coefficients for color, not RGB. Most loaders convert to display color with rgb = 0.5 + SH_C0 * f_dc, where SH_C0 = 0.28209479177387814.
• opacity is stored as a logit. Viewers map it with alpha = 1 / (1 + exp(-opacity)).
• scale_0..2 are log scales. Viewers map them with sigma = exp(scale_i).
  If you wrote “linear” values here, the result will look wrong or the shader path might blow up. ([thomasantony.com]2)

Finally, Babylon and PlayCanvas now support several splat formats. If you give them a PLY that matches the expected Gaussian schema, they load fine. If you give them a generic PLY from CloudCompare or MeshLab, they do not, because those tools export mesh or point attributes, not Gaussian attributes. ([Babylon.js Docs]3)

---

A reliable pipeline from PCD to Gaussian PLY

You have two choices. The first is the quick pragmatic route that makes your PCD “look” like a Gaussian PLY with reasonable defaults. The second is the high quality route that actually fits anisotropic Gaussians.

Option A: Minimal conversion that works in web viewers

Use your PCD positions and colors, then synthesize the missing Gaussian fields:

1. Read PCD and get xyz and rgb in [0, 1].

2. Map color to DC SH coefficients:
   f_dc = (rgb - 0.5) / SH_C0, where SH_C0 = 0.28209479177387814. Set all f_rest_* to 0 if you do not want view dependent color. ([Medium]4)

3. Choose a constant alpha like 0.6 and store opacity = log(alpha / (1 - alpha)). Viewers will sigmoid it back. ([thomasantony.com]2)

4. Estimate a reasonable isotropic size. A common heuristic is the median distance to the k nearest neighbors times a factor like 0.5 to 1.5. Store log scales:
   s = log(sigma) and write the same s into scale_0, scale_1, scale_2. ([thomasantony.com]2)

5. Use identity orientation for each splat. Most web stacks expect a quaternion rot_0..3 in WXYZ order, so write rot = (1, 0, 0, 0). Some older code paths accepted a 3 value representation, but PlayCanvas and many others document a 4 value quaternion. ([PlayCanvas Blog]5)

6. Optionally write normals nx ny nz. If you do not have them, some loaders do not use them, so you can set zeros.

7. Write a binary little endian PLY with the exact property names and order shown above.

8. If the model appears upside down in some engines, apply a transform or flip Y during export. This is a known alignment quirk across engines and file flavors. ([Babylon.js]6)

If you just want a viewer friendly file and do not need SH color, you can also convert the PLY you produced into a compressed variant that the web engines like for performance.

• Use SplatTransform to convert and sanity check. It reads PLY and writes PLY or compressed PLY and can filter NaNs or bands.
  Example:
  splat-transform input.ply --filterNaN output.compressed.ply
  It also converts to PlayCanvas SOG and can generate a small HTML viewer. ([GitHub]7)

• For the Three.js demo by Mark Kellogg, converting to .ksplat also works great and is faster to load. The repo and demo accept INRIA style PLY and compressed formats. ([GitHub]8)

Option B: Higher quality conversion

If you actually want anisotropic Gaussians that match the surface, you need per point covariance and orientation, not only a guessed isotropic size. That usually means:

• Estimate local covariance from neighbors, then eigen decompose to get axes and scales.
• Convert eigenvalues to log scales, and eigenvectors to a quaternion for rot_0..3.
• Keep f_rest_* if you have SH color from a trained scene. For a raw PCD you probably do not, so leave them zero.

This produces much better results than isotropic blobs but takes more preprocessing.

---

A small end to end sketch in Python

This is the skeleton you can adapt inside your FastAPI route. It reads a PCD, synthesizes missing Gaussian fields, and writes a valid PLY. It uses Open3D for I/O and KNN, and plyfile to emit a correct binary header.

import numpy as np
import open3d as o3d
from plyfile import PlyData, PlyElement

SH_C0 = 0.28209479177387814

def to_logit(a):
    a = np.clip(a, 1e-6, 1-1e-6)
    return np.log(a/(1-a))

def estimate_sigma(points, k=8):
    pcd = o3d.geometry.PointCloud()
    pcd.points = o3d.utility.Vector3dVector(points)
    kdtree = o3d.geometry.KDTreeFlann(pcd)
    sigmas = np.zeros(len(points), dtype=np.float32)
    for i, p in enumerate(points):
        _, idx, dists2 = kdtree.search_knn_vector_3d(p, k)
        # skip distance to self at index 0
        d = np.sqrt(np.maximum(dists2[1:], 1e-12))
        sigmas[i] = np.median(d) * 0.75
    return sigmas

def pcd_to_gauss_ply(pcd_path, ply_out, alpha=0.6):
    pcd = o3d.io.read_point_cloud(pcd_path)
    xyz = np.asarray(pcd.points, dtype=np.float32)

    if pcd.has_colors():
        rgb = np.asarray(pcd.colors, dtype=np.float32)
    else:
        rgb = np.ones_like(xyz) * 0.5  # gray

    # DC SH from color
    f_dc = (rgb - 0.5) / SH_C0

    # No view dependent terms
    f_rest = np.zeros((len(xyz), 45), dtype=np.float32)  # degree 3 -> 45 terms

    # opacities in logit space
    opacity = np.full((len(xyz), 1), to_logit(alpha), dtype=np.float32)

    # isotropic size from KNN, stored as log scales
    sigma = estimate_sigma(xyz)
    s = np.log(np.maximum(sigma, 1e-6)).astype(np.float32)
    scales = np.stack([s, s, s], axis=1)

    # identity quaternion WXYZ
    rot = np.tile(np.array([1.0, 0.0, 0.0, 0.0], dtype=np.float32), (len(xyz), 1))

    # normals optional -> zeros
    nrm = np.zeros_like(xyz, dtype=np.float32)

    # interleave into structured array matching the header order
    dtype = [
        ('x', 'f4'), ('y', 'f4'), ('z', 'f4'),
        ('nx','f4'), ('ny','f4'), ('nz','f4'),
        ('f_dc_0','f4'), ('f_dc_1','f4'), ('f_dc_2','f4')
    ] + [(f'f_rest_{i}', 'f4') for i in range(45)] + [
        ('opacity','f4'),
        ('scale_0','f4'), ('scale_1','f4'), ('scale_2','f4'),
        ('rot_0','f4'), ('rot_1','f4'), ('rot_2','f4'), ('rot_3','f4')
    ]
    arr = np.empty(len(xyz), dtype=dtype)
    arr['x'], arr['y'], arr['z'] = xyz[:,0], xyz[:,1], xyz[:,2]
    arr['nx'], arr['ny'], arr['nz'] = nrm[:,0], nrm[:,1], nrm[:,2]
    arr['f_dc_0'], arr['f_dc_1'], arr['f_dc_2'] = f_dc[:,0], f_dc[:,1], f_dc[:,2]
    for i in range(45):
        arr[f'f_rest_{i}'] = f_rest[:, i]
    arr['opacity'] = opacity[:,0]
    arr['scale_0'], arr['scale_1'], arr['scale_2'] = scales[:,0], scales[:,1], scales[:,2]
    arr['rot_0'], arr['rot_1'], arr['rot_2'], arr['rot_3'] = rot[:,0], rot[:,1], rot[:,2], rot[:,3]

    el = PlyElement.describe(arr, 'vertex')
    PlyData([el], text=False).write(ply_out)

Drop this into your FastAPI handler after you receive the PCD bytes. Return the resulting .ply file. This produces a standards compliant Gaussian PLY that will load in the viewers in your list. The color will be flat lit since f_rest_* are zero, which is expected when you convert a raw point cloud.

---

Engine specific tips

• PlayCanvas documents the Gaussian PLY layout very clearly and provides CLI tooling (SplatTransform) for conversion, compression, filtering, and quick HTML previews. If you hit orientation issues, rotate or invert Y with the tool, then re export. ([PlayCanvas Blog]5)

• Babylon.js supports PLY, compressed PLY, and Niantic SPZ. If a file that worked in PlayCanvas fails in Babylon, it is often due to orientation, SH band settings, or a known regression in a specific engine version. Try their latest 8.x and flip Y as a quick test. ([Babylon.js Docs]3)

• Three.js demo by Mark Kellogg accepts INRIA style PLY and also a fast .ksplat format. For very large scenes, convert to .ksplat to avoid progressive PLY overhead. ([GitHub]8)

---

Quick checklist for your FastAPI converter

1. Write binary little endian and match property names and order exactly. ([GitHub]1)
2. Encode values, do not store linear ones: f_dc, opacity logit, scale log. ([thomasantony.com]2)
3. Provide a four component quaternion rot_0..3. Use (1, 0, 0, 0) if you do not compute orientation. ([PlayCanvas Blog]5)
4. Remove NaN and Inf. SplatTransform has a --filterNaN pass that is handy. ([GitHub]7)
5. Keep counts in check. Web viewers can handle millions, but huge uncompressed PLYs will stall parsing. Convert to .compressed.ply, .ksplat, or PlayCanvas .sog for faster loads. ([GitHub]7)

If you follow the steps above you should be able to import your converted files into all three links you posted without the browser falling over.