Deep learning approach to generate 3D civil infrastructure models using drone images

다국어 초록 (Multilingual Abstract)

Three-dimensional (3D) models have become crucial for improving civil infrastructure analysis, and they can be used for various purposes such as damage detection, risk estimation, resolving potential safety issues, alarm detection, and structural health monitoring. 3D point cloud data is used not only to make visual models but also to analyze the states of structures and to monitor them using semantic data. This study proposes automating the generation of high-quality 3D point cloud data and removing noise using deep learning algorithms. In this study, large-format aerial images of civilian infrastructure, such as cut slopes and dams, which were captured by drones, were used to develop a workflow for automatically generating a 3D point cloud model. Through image cropping, downscaling/upscaling, semantic segmentation, generation of segmentation masks, and implementation of region extraction algorithms, the generation of the point cloud was automated. Compared with the method wherein the point cloud model is generated from raw images, our method could effectively improve the quality of the model, remove noise, and reduce the processing time. The results showed that the size of the 3D point cloud model created using the proposed method was significantly reduced; the number of points was reduced by 20-50%, and distant points were recognized as noise. This method can be applied to the automatic generation of high-quality 3D point cloud models of civil infrastructures using aerial imagery.

참고문헌 (Reference)

1 Ali Khaloo, "Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling" Informa UK Limited 14 (14): 124-136, 2017

2 Irschara, A., "Towards fully automatic photogrammetric reconstruction using digital images taken from UAVs" 38 (38): 65-70, 2010

3 Hany Omar, "Towards an automated photogrammetry-based approach for monitoring and controlling construction site activities" Elsevier BV 98 : 172-182, 2018

4 Hafizur Rahaman, "To 3D or Not 3D: Choosing a Photogrammetry Workflow for Cultural Heritage Groups" MDPI AG 2 (2): 1835-1851, 2019

5 "The Cityscapes Dataset"

6 Mustafa, A., "Temporally coherent general dynamic scene reconstruction" 129 (129): 123-141, 2021

7 Ziwen Liu, "Semantic segmentation and photogrammetry of crowdsourced images to monitor historic facades" Springer Science and Business Media LLC 10 (10): 2022

8 Chen, L. C., "Semantic image segmentation with deep convolutional nets and fully connected crfs"

9 Fusheng Zha, "Semantic 3D Reconstruction for Robotic Manipulators with an Eye-In-Hand Vision System" MDPI AG 10 (10): 1183-, 2020

10 Chen, L. C., "Rethinking atrous convolution for semantic image segmentation"

11 Tao He, "Quantifying spatial distribution of interrill and rill erosion in a loess at different slopes using structure from motion (SfM) photogrammetry" Elsevier BV 10 (10): 393-406, 2022

12 "Pix4D"

13 Samir El-Omari, "Integrating 3D laser scanning and photogrammetry for progress measurement of construction work" Elsevier BV 18 (18): 1-9, 2008

14 Laura Inzerillo, "Image-based 3D reconstruction using traditional and UAV datasets for analysis of road pavement distress" Elsevier BV 96 : 457-469, 2018

15 Shervin Minaee, "Image Segmentation Using Deep Learning: A Survey" Institute of Electrical and Electronics Engineers (IEEE) 1-1, 2021

16 Shengjun Tang, "Fast and Automatic Reconstruction of Semantically Rich 3D Indoor Maps from Low-quality RGB-D Sequences" MDPI AG 19 (19): 533-, 2019

17 Chen, L. C., "Encoder-decoder with atrous separable convolution for semantic image segmentation" 2018

18 Kaan Yücer, "Efficient 3D Object Segmentation from Densely Sampled Light Fields with Applications to 3D Reconstruction" Association for Computing Machinery (ACM) 35 (35): 1-15, 2016

19 Yuhan Jiang, "Determining Ground Elevations Covered by Vegetation on Construction Sites Using Drone-Based Orthoimage and Convolutional Neural Network" American Society of Civil Engineers (ASCE) 34 (34): 2020

20 Niccolò Menegoni, "Detection and geometric characterization of rock mass discontinuities using a 3D high-resolution digital outcrop model generated from RPAS imagery – Ormea rock slope, Italy" Elsevier BV 252 : 145-163, 2019

21 Liang-Chieh Chen, "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs" Institute of Electrical and Electronics Engineers (IEEE) 40 (40): 834-848, 2018

22 Tao Liu, "Deep convolutional neural network training enrichment using multi-view object-based analysis of Unmanned Aerial systems imagery for wetlands classification" Elsevier BV 139 : 154-170, 2018

23 "COLMAP"

24 "Agisoft Metashape Standard Edition"

25 Cosmin Popescu, "3D reconstruction of existing concrete bridges using optical methods" Informa UK Limited 15 (15): 912-924, 2019