Real-time 3D reconstruction with hand-held scanners

During my semester abroad at the Technical University of Munich in 2020/2021, I collaborated with two graduate students on implementing a real-time 3D reconstruction system for hand-held depth scanners. The full report (PDF) has all the details.

Abstract

In this report, we first introduce the KinectFusion approach and why it is interesting to implement. Afterward, an overview of the method is given. The main part discusses different implementation details, as well as the GUI and marching cubes implementation. Finally, we briefly go over experimental results and give a conclusion.

KinectFusion pipeline: measurement, pose estimation with ICP, reconstruction update into a global TSDF, and surface prediction by ray casting, with depth-map and model previews — The KinectFusion pipeline: surface measurement, ICP pose estimation, TSDF fusion and ray-cast surface prediction.

Table of frame rates and GPU memory for 256³, 384³ and 512³ volumes, and a bar chart showing that fusion takes about 70% of the runtime — Frame rate and GPU memory by volume size on the freiburg_xyz dataset (Ryzen 5600X, GTX 1070), and the runtime share of each kernel for a 512³ grid.