Nebula: Infinite-Scale 3D Gaussian Splatting in Virtual Reality via Collaborative Rendering and Accelerated Stereo Rasterization

Abstract

3D Gaussian splatting (3DGS) has drawn significant attention in the architectural community recently. However, current architectural designs often overlook the 3DGS scalability, making them fragile for extremely large-scale 3DGS. Meanwhile, the VR bandwidth requirement makes it impossible to deliver high-fidelity and smooth VR content from the cloud.

We present Nebula, a coherent acceleration framework for large-scale 3DGS collaborative rendering. Instead of streaming videos, Nebula streams intermediate results after the LoD search, reducing 1925% data communication between the cloud and the client. To further enhance the motion-to-photon experience, we introduce a temporal-aware LoD search in the cloud that tames the irregular memory access and reduces redundant data access by exploiting temporal coherence across frames. On the client side, we propose a novel stereo rasterization that enables two eyes to share most computations during the stereo rendering with bit-accurate quality. With minimal hardware augmentations, Nebula achieves 2.7$\times$ motion-to-photon speedup and reduces 1925% bandwidth over lossy video streaming.

Contribution

We introduce our collaborative rendering framework Nebula, which orchestrate the compute resources of both the cloud and the client. On the cloud side, we first perform the temporal-aware LoD search and update the Gaussian management system. The cloud then compresses the Gaussians that need to be transmitted and sends them to the client over the network. On the client side, we first decompress the data upon arrival. Meanwhile, the client continuously traverses the subgraph and the remaining stages of the 3DGS rendering. Once rendering completes, the new image is displayed at the next VSync arrival.

We specifically design the components in our framework to minimize the data communication and rendering latency. First, we propose a temporal-aware LoD search algorithm that exploits the temporal coherence across frames to reduce the redundant data access and mitigate the irregular memory access in the LoD search stage. Next, our runtime gaussian management system tracks Gaussians on the client side and only transmits the Gaussians that are necessary for rendering the current frame, significantly reducing the data communication between the cloud and the client. Lastly, we design a novel stereo rasterization algorithm that enables two eyes to share most computations during the stereo rendering with bit-accurate quality, further reducing the rendering latency on the client side.

The stereo rasterization algorithm leverage the stereo similarity between the two eyes and reuse the contribution of each Gaussian to both eyes to avoid the redundant preprocessing and $\alpha$-checking. We further add architechtural support for stereo rasterization, which forwards the contribution of each Gaussian to the other eye and pipelines all rendering stages.

Results

Rendering Quality

Hardware Performance

More videos

We provide some GPU performance comparisons between (left) Hierarchical Gaussian Splatting and (right) Nebula to show strengths of our method in different scenarios.

BibTeX

@inproceedings{zhu2026nebula,
  title={Nebula: Infinite-Scale 3D Gaussian Splatting in VR via Collaborative Rendering and Accelerated Stereo Rasterization},
  author={Zhu, He and Liu, Zheng and Li, Xingyang and Wu, Anbang and Zhao, Jieru and Liu, Fangxin and Gan, Yiming and Leng, Jingwen and Feng, Yu},
  booktitle={Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
  pages={1235--1250},
  year={2026}
}

Acknowledgements

We are sincerely grateful to Weikai Lin for providing valuable advice and support throughout this research.