MFNeRF: Memory Efficient NeRF with Mixed-Feature Hash Table

Abstract

Recently neural radiance fields (NeRFs) have shown remarkable performance in generating photorealistic novel views in 3D modeling. The traditional NeRF typically requires extensive training and long rendering times inspiring many recent works to utilize efficient data structures such as feature grids to ease the computational burden of multilayer perceptron networks. However Those approaches require storing features in dense grids that demands a substantial amount of memory space resulting in a notable memory bottleneck. Consequently it leads to a significant increase in training time. To address this issue in this work we propose MFNeRF a memory-efficient NeRF framework that employs a mixed-feature hash table to improve memory efficiency and reduce training time while maintaining reconstruction quality. Specifically we first design a mixed-feature hash encoding method to adaptively mix parts of multi-level feature grids and map them to a single hash table. Following that in order to obtain the correct index of a grid point we further develop an index transformation method that transforms indices of an arbitrary-level grid to those of a canonical grid. Extensive benchmarking against the state-of-the-art methods including InstantNGP TensoRF and DVGO indicates that our MFNeRF achieves faster training and rendering times on the same GPU hardware with a significantly smaller memory while maintaining similar or even higher reconstruction quality. Compared to the InstantNGP-Big model our method could achieve 89% improvement in the figure of merit defined in terms of PSNR*FPS/MB.