NeRF와 이미지 인페인팅을 이용한 가려진 객체의 표면 재구성

In traditional 3D vision research, the most representative 3D reconstruction method is to use Multiple View Geometry. Since the Multiple View Geometry method requires the extract of the target's feature points, it requires clear feature objects and a large number of images for a smooth reconstruction. Recent advances in deep learning have provided ways to address these constraints. In particular, 3D reconstruction has been advancing rapidly since the publication of NeRF. However, since the NeRF method uses a given camera view to learn 3D, it does not restore occluded areas cleanly. In this study, we use the Image Inpainting technique to restore the occluded part of the target on the 2D image, and then restore the occluded part to 3D during the NeRF learning process. This resulted in a PSNR performance improvement of about 46% over training the network using the occluded image.