Our paper title “Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation” has been accepted for publication at the Winter Conference on Applications of Computer Vision (WACV).
The paper tackles the challenging real-world task to semantically segment cracks in concrete infrastructure. However, rather than relying on an abundance of manually labelled data, the paper overcomes the hurdle of manual annotation and fragile data-driven learning in low-data regimes by leveraging expert knowledge and simulation. As such, it first introduces a fractal-based physics based simulation pipeline for data generation, and then combines a deep learner on this data with a topological pit estimator to exploit domain knowledge. The resulting system learns exclusively through simulation, yet is shown to beat state-of-the-art approaches that are trained on a variety of real-world data in respective real-world concrete segmentation tasks.
For more information, read the full paper or see the abstract below:
Identification of cracks is essential to assess the structural integrity of concrete infrastructure. However, robust crack segmentation remains a challenging task for computer vision systems due to the diverse appearance of concrete surfaces, variable lighting and weather conditions, and the overlapping of different defects. In particular recent data-driven methods struggle with the limited availability of data, the fine-grained and time-consuming nature of crack annotation, and face subsequent difficulty in generalizing to out-of-distribution samples. In this work, we move past these challenges in a two-fold way. We introduce a high-fidelity crack graphics simulator based on fractals and a corresponding fully-annotated crack dataset. We then complement the latter with a system that learns generalizable representations from simulation, by leveraging both a pointwise mutual information estimate along with adaptive instance normalization as inductive biases. Finally, we empirically highlight how different design choices are symbiotic in bridging the simulation to real gap, and ultimately demonstrate that our introduced system can effectively handle real-world crack segmentation.
Abstract of our paper “Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation”