Instance Deformable Metal Splatter

Code

Our on going project on instance aware dynamic 3D gaussian splatting on Apple devices, based on my friends (Li et. al.) paper TRASE [3DV 2026]

This project is an extended fork of Metal Splatter which allows to render 3D Gaussian Splats on Apple Devices with Metal support. My friend extended this framework to support dynamic 3D Gaussian Splatting by running the deformation network of Deformable-3DGS directly in Metal on Apple devices.

I later added support for instance aware gaussian splats by integrating the TRASE cluster support which enables cluster visualisation and cluster based selection of gaussian splats. Additionally, the deformation network is accelerated via optional FP16 support yielding about 2x faster inference speed.

For more details please see the repository which is more up to date. We aim to further speed up the pipeline (e.g. by masking dynamic vs static splats beforehand for faster inference of the deformation network) or extending it with new features (such as style transfer, text based object selection, etc.).

Please also see the TRASE paper for more details on the instance aware deformable 3D gaussian splatting and the TRASE repo.

Visualisations

Results on iPad (Cookie) and iPhone (Steak).

References

  1. Yun-Jin Li, Mariia Gladkova, Yan Xia, and Daniel Cremers. TRASE: Tracking-free 4D Segmentation and Editing. In Proceedings of the International Conference on 3D Vision (3DV), 2026.

  2. Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for highfidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024.