Neural radiance fields with transformer rendering

The existing Neural Radiance Fields (NeRF) uses deep learning only for the part that infers density during the sampling-network-rendering process. It is known that learning by configuring end-to-end can achieve better performance than learning only part of it. This paper proposes a transformer rendering technique that does not require multi-views, unlike the existing NeRF method using a transformer structure. The structure is built to mimic the volume rendering technique used in NeRF and can be used as a replacement for volume rendering. When used, the network shows color composition ability similar to NeRF and about 6% improvement in depth inference ability. In particular, it shows that the depth inference ability is about 35% better, even when the neural field network is small. Although it has the disadvantage of increasing learning parameters, it has the advantage of reducing actual memory usage by about 36%. This indicates that the rendering process can be sufficiently replaced with a network.