3D Scene Understanding
This project page gives an overview over the research done on the topic of 3D Scene Understanding, where we tackle two challenging problems; point cloud classification and contour extraction.
Large-Scale Point Cloud Classification Benchmark - external pagewww.semantic3d.net
3D point cloud classification is an important task with applications in robotics, augmented reality and urban planning. Recent advances in Machine Learning and Computer Vision have proven that complex real-world tasks require large training data sets for classifier training. At the same time, until now there were no data sets for 3D point cloud classification which would be sufficiently rich in both object representations and number of labelled points. For example, the well-known Oakland data set contains less than 2 million labelled points. Another popular data set, the NYU benchmark, provides only indoor scenes. Finally, both Sydney Urban Objects data set and the IQmulus & TerraMobilita Contest use a 3D Velodyne LIDAR mounted on a car which provides much lower point density than a static scanner. The same counts for the Vaihingen3D airborne benchmark.
This benchmark closes the gap and provides the largest known labelled 3D point cloud data set of natural scenes with over 3 billion points in total. It also covers a range of diverse urban scenes: churches, streets, railroad tracks, squares, villages, soccer fields, castles to name just a few. The point clouds we provide are scanned statically with state-of-the-art equipment and contain very fine details. Our goal is to help data-demanding methods like deep neural nets to unleash their full power and to learn richer 3D representations than it was ever possible before.
Fast semantic segmentation of 3D point clouds with strongly varying density
Timo Hackel, Jan D. Wegner, Konrad Schindler
Abstract:
We describe an effective and efficient method for point-wise semantic classification of 3D point clouds. The method can handle unstructured and inhomogeneous point clouds such as those derived from static terrestrial LiDAR or photogammetric reconstruction; and it is computationally efficient, making it possible to process point clouds with many millions of points in a matter of minutes. The key issue, both to cope with strong variations in point density and to bring down computation time, turns out to be careful handling of neighborhood relations. By choosing appropriate definitions of a point’s (multi-scale) neighborhood, we obtain a feature set that is both expressive and fast to compute. We evaluate our classification method both on benchmark data from a mobile mapping platform and on a variety of large, terrestrial laser scans with greatly varying point density. The proposed feature set outperforms the state of the art with respect to per-point classification accuracy, while at the same time being much faster to compute.
Timo Hackel, Jan D. Wegner, Konrad Schindler: Fast semantic segmentation of 3d point clouds with strongly varying density (PDF, 5.6 MB). ISPRS Annals - ISPRS Congress, Prague, 2016
Contour detection in unstructured 3D point clouds
Timo Hackel, Jan D. Wegner, Konrad Schindler
Abstract:
We describe a method to automatically detect contours, i.e. lines along which the surface orientation sharply changes, in large-scale outdoor point clouds. Contours are important intermediate features for structuring point clouds and converting them into high-quality surface or solid models, and are extensively used in graphics and mapping applications. Yet, detecting them in unstructured, inhomogeneous point clouds turns out to be surprisingly difficult, and existing line detection algorithms largely fail. We approach contour extraction as a two-stage discriminative learning problem. In the first stage, a contour score for each individual point is predicted with a binary classifier, using a set of features extracted from the point’s neighborhood. The contour scores serve as a basis to construct an overcomplete graph of candidate contours. The second stage selects an optimal set of contours from the candidates. This amounts to a further binary classification in a higher-order MRF, whose cliques encode a preference for connected contours and penalize loose ends. The method can handle large point clouds with > 10.000.000 points in a couple of minutes, and vastly outperforms a baseline that performs Canny-style edge detection on a range image representation of the point cloud.
Timo Hackel, Jan D. Wegner, Konrad Schindler: Contour detection in unstructured 3D point clouds (PDF, 1.4 MB), IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016
Contact Details: Timo Hackel