Projects

PhD

Projects I have worked on during my PhD at TU Munich

SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans (CVPR'20)
We present a novel approach that converts partial and noisy RGB-D scans into high-quality 3D scene reconstructions by inferring unobserved scene geometry.
Our approach is fully self-supervised and can hence be trained solely on real-world, incomplete scans.
To achieve self-supervision, we remove frames from a given (incomplete) 3D scan in order to make it even more incomplete; self-supervision is then formulated by correlating the two levels of partialness of...


Master

Projects I have worked on during my Master's studies at TU Munich

3D Shape Completion from Sparse Point Clouds Using Deep Learning
We tackle the problem of generating dense representations from sparse and partial point clouds. We achieve this with a data-driven approach which learns to complete incoming sets of 3D points in a fully-supervised manner.
To this end, we first prepare a suitable dataset containing partial scans, each one correlated with complete ground-truth representations. We base this on the widely used ModelNet40 dataset which features high-quality CAD scans of 40 different object...

BundleACeres - Structure from Motion using Bundle Adjustment with the Ceres Solver
An implementation of Bundle Adjustment with the Ceres Solver as optimizer. It matches a chain of incoming RGB images using ORB keypoints and BRISK descriptors in order to find 3D points. The Ceres Solver is then used to jointly solve for the locations of 3D points and camera poses of the frames. It is implemented in modern C++14 and utilizes OpenCV 3, Ceres, PCL and others. For more information, feel free to have a look at the project page.

KinectFusion - Real-Time depth image fusion and dense surface mesh extraction
Implementation of the KinectFusion approach to generating three-dimensional models from depth image scans. Here, the original method has been extended with the MarchingCubes algorithm to allow exporting the model as a dense surface mesh. Realized in modern C++14 and CUDA to allow real-time reconstruction. Developed in the context of an interdisciplinary project in cooperation with the Chair for Computer Aided Medical Procedures and Augmented Reality and Dynamify. Feel free to have a look at...


Bachelor

Projects I have worked on during my Bachelor's studies at the University of Augsburg

VideoMagnification - Magnify motions and detect heartbeats
This application allows the user to magnify motion and detect heartbeats from videos and webcam video streams. It is an implementation of Wu, Hao-Yu, et al.: “Eulerian video magnification for revealing subtle changes in the world”. You can find out more about motion magnification on the project homepage. My implementation can be found here. It makes use of modern C++11, the open-source vision library OpenCV 3.1 and the UI framework QT 5.

Spatial Pyramid Pooling as an additional layer in caffe, the Deep Learning Framework
An implementation of the concept proposed in He, Kaiming, et al. “Spatial pyramid pooling in deep convolutional networks for visual recognition“.
It adds a new custom layer to the AlexNet architecture which performs spatial pyramid pooling in order to remove the network’s need for fixed-size input images.
This was part of my Bachelor’s Thesis which I wrote at the Multimedia Computing and Computer Vision Lab, University of Augsburg under the supervision...

Training methods for the decision forest library fertilized forests
Worked with Christoph Lassner on his library fertilized forests and implemented several boosted training methods like AdaBoost and SAMME. You can have a look at the project homepage or read the paper which received an honorable mention at the ACM MM OSSC 2015.