Data Orchestration for Accelerating GPU-Based Light Field Rendering Aiming at a Wide Virtual Space

Recently, research on six-degree-of-freedom virtual reality (VR) systems based on a light field (LF) has been actively conducted. The LF-based approach is photorealistic and less prone to errors than the existing three-dimensional (3D) modeling-based approach. On the other hand, because the amount of data is very large for LF-based immersive virtual reality, it is important to manage the data efficiently. This paper makes two major contributions to data management for rendering acceleration in this context. First, the GPUs commonly used for high-speed rendering require optimization of the data transfer process between the host memory and the device memory. This paper proposes a simple but effective way to maximize data reuse by finding the proper size of a transfer unit depending on the target applications and the system capability constraints. Also, view synthesis and data transfer are performed in a pipelined manner to hide the data movement time. Second, for the LF rendering of a wide space, data management in the storage-host-GPU hierarchy is attempted for the first time. As the size of the virtual space increases, the required amount of LF data grows rapidly and exceeds the allocatable host memory capacity. Because the loading of data stored in storage is a very slow process, a system-level LF rendering structure that carefully considers the characteristics of the storage-host-GPU memory hierarchy should be designed. In this paper, progressive LF updating is proposed, and it prevents rendering stalls caused by slow storage data loading. Experimental results show that a 360-degree view with a 9000 × 2048 resolution can be rendered in an average of 2.57 ms from LF data with a $4096 \times 2048$ resolution, achieving performance close to 400 frames per second.