Generating 3D texture models of vessel pipes using 2D texture transferred by object recognition

다국어 초록 (Multilingual Abstract)

Research and development of smart vessels has progressed significantly in recent years, and ships have become high-value technology-intensive resources. These ships entail high production costs and long-life cycles. Thus, modernized technical design, professional training, and aggressive maintenance are important factors in the efficient management of ships. With the continuing digital revolution, the industrial shipbuilding applicability of augmented reality (AR) and virtual reality (VR) technologies as well as related 3D system modeling and processes has increased. However, resolving the differences between AR/VR and real-world models remains burdensome. This problem is particularly evident when mapping various texture characteristics to virtual objects. To mitigate the burden and improve the performance of such technologies, it is necessary to directly define various texture characteristics or to express them using expensive equipment. The use of deep-learning-based CycleGAN, however, has gained attention as a method of learning and automatically mapping real-object textures. Thus, we seek to use CycleGAN to improve the immersive capacities of AR/VR models and to reduce production costs for shipbuilding. However, when applying CycleGAN’s textures to pipe structures, the performance is insufficient for direct application to industrial piping networks. Therefore, this study investigates an improved CycleGAN algorithm that can be specifically applied to the shipbuilding industry by combining a modified object-recognition algorithm with a double normalization method. Thus, we demonstrate that basic knowledge on the production of AR industrial pipe models can be applied to virtual models through machine learning to deliver low-cost and high-quality textures. Our results provide an on-ramp for future CycleGAN studies related to the shipbuilding industry.

참고문헌 (Reference)

1 김민지, "증강현실에서의 현실감 있는 가상 모델 가시화를 위한 Texture의 생성에 관한 비교 연구" 한국CDE학회 24 (24): 169-179, 2019

2 김승현, "선박해양구조물의 유지보수 단계 수명주기관리 정보 시스템의 개념 설계와 구현" 한국해양공학회 26 (26): 58-67, 2012

3 길우성, "선박 검사 교육훈련을 위한 VR 선박 환경 구축" 대한조선학회 55 (55): 361-369, 2018

4 Cebollero, A., "Virtual reality empowered design" 2019

5 Liu, M. Y., "Unsupervised image-toimage translation networks" 2017

6 Cao, Y., "Unsupervised diverse colorization via generative adversarial networks" 10534 : 151-166, 2017

7 Zhu, J. Y., "Unpaired imageto-image translation using cycle-consistent adversarial networks" 2017

8 Welsh, T., "Transferring color to greyscale images" 277-280, 2002

9 Ovcharenko, O., "Style transfer for generation of realistically textured subsurface models" 2393-2397, 2019

10 Choi, Y., "StarGAN: Unified generative adversarial networks for multidomain image-to-image translation" 8789-8797, 2018

11 Jakub, W., "Ship information systems using smartglasses technology" 100 : 211-222, 2018

12 Buyukdemircioglu, M., "Semiautomatic 3D city model generation from large-format aerial images" 7 (7): 339-, 2018

13 Odena, A., "Semi-Supervised learning with generative adversarial networks" 2016

14 Chavdarova, T., "SGAN: An alternative training of generative adversarial networks" 9407-9415, 2017

15 Won-Hyuk Lee, "Registration method for maintenance-work support based on augmented-reality-model generation from drawing data" 한국CDE학회 7 (7): 775-787, 2020

16 Heindl, C., "Photorealistic texturing of human busts reconstruction" 225-230, 2016

17 Yifan Yang, "Mesh processing for improved perceptual quality of 3D printed relief" 한국CDE학회 8 (8): 115-124, 2021

18 Alessandro Ceruti, "Maintenance in aeronautics in an Industry 4.0 context: The role of Augmented Reality and Additive Manufacturing" 한국CDE학회 6 (6): 516-526, 2019

19 Mao, X., "Least squares generative adversarial networks" 1 : 2813-2821, 2017

20 Kim, T., "Learning to discover cross-domain relations with generative adversarial networks"

21 Montwitt, A., "Importance of key phases of the ship manufacturing system for efficient vessel life cycle management" 19 : 34-41, 2018

22 Isola, P., "Image-to-image translation with conditional adversarial networks" 2016

23 Hertzmann, A., "Image analogies" 327-340, 2001

24 Bernardini, F., "High-quality texture reconstruction from multiple scans" 7 (7): 318-332, 2001

25 Wu, Y., "Group normalization" 2018

26 Goodfellow, I., "Generative Adversarial Networks" 2016

27 Saddik, A., "Digital Twins : The convergence of multimedia technologies" 25 (25): 87-92, 2018

28 Kim, D., "Development of an ARbased method for augmentation of 3D CAD data onto a real ship block image" 98 : 1-11, 2018

29 Hikida, K., "Development of a visual lookout support system with headup display" 122 : 7-13, 2010

30 He, K., "Deep residual learning for image recognition" 770-778, 2015

31 Deng, L., "Deep learning: Methods and application" 7 (7): 197-387, 2014

32 Goodfellow, I., "Deep learning" MIT Press 2016

33 Nam, H., "Batch-instance normalization for adaptively style-invariant neural networks" 2018

34 Alshawabkeh, Y., "Automatic multi-image photo-texturing of complex 3D scenes" 34 : 1-6, 2005

35 Liniger, S., "Automatic method for photo texturing geolocated 3D models from geolocated imagery"

36 Laycock, R. G., "Automatic generation, texturing and population of a reflective real-time urban environment" 31 (31): 625-635, 2007

37 Huang, H., "An introduction to image synthesis with generative adversarial nets"

38 Tomohiro Fukuda, "An indoor thermal environment design system for renovation using augmented reality" 한국CDE학회 6 (6): 179-188, 2019

39 Ke, S., "An enhanced interaction framework based on VR, AR and MR in Digital Twin" 83 : 753-758, 2019

40 Fraga-Lamas, P., "A review on industrial augmented reality systems for the industry 4. 0 shipyard" 6 : 13358-13375, 2018

41 Harrison, P., "A non-hierarchical procedure for re-synthesis of complex texture" University of West Bohemia 190-197, 2001

42 Gatys, L. A., "A neural algorithm of artistic style" 16 (16): 326-, 2016

43 "3Dmaritim"

44 Chen, Y., "3D texture mapping for rapid manufacturing" 4 (4): 761-771, 2007