가상 환경에서의 강화학습을 활용한 모바일 로봇의 장애물 회피 = Obstacle Avoidance of Mobile Robot Using Reinforcement Learning in Virtual Environment

다국어 초록 (Multilingual Abstract)

In order to apply reinforcement learning to a robot in a real environment, it is necessary to use simulation in a virtual environment because numerous iterative learning is required. In addition, it is difficult to apply a learning algorithm that requires a lot of computation for a robot with low-spec. hardware. In this study, ML-Agent, a reinforcement learning frame provided by Unity, was used as a virtual simulation environment to apply reinforcement learning to the obstacle collision avoidance problem of mobile robots with low-spec hardware. A DQN supported by ML-Agent is adopted as a reinforcement learning algorithm and the results for a real robot show that the number of collisions occurred less then 2 times per minute.

국문 초록 (Abstract)

실 환경에서 로봇에 강화학습을 적용하기 위해서는 수많은 반복 학습이 필요하므로 가상 환경에서의 시뮬레이션을 사용할 수밖에 없다. 또한 실제 사용하는 로봇이 저사양의 하드웨어를 가지고 있는 경우 계산량이 많은 학습 알고리즘을 적용하는 것은 어려운 일이다. 본 연구에서는 저사양의 하드웨어를 가지고 있는 모바일 로봇의 장애물 충돌 회피 문제에 강화학습을 적용하기 위하여 가상의 시뮬레이션 환경으로서 Unity에서 제공하는 강화학습 프레임인 ML-Agent를 활용하였다. 강화학습 알고리즘으로서 ML-Agent에서 제공하는 DQN을 사용하였으며, 이를 활용하여 학습한 결과를 실제 로봇에 적용해 본 결과 1분간 충돌 횟수가 2회 이하로 발생하는 결과를 얻을 수 있었다.

참고문헌 (Reference)

1 이동우, "사물인터넷 환경에서 자율주행 기반의 주요국 드론 특성 비교/분석" 한국사물인터넷학회 6 (6): 31-36, 2020

2 "https://github.com/Unity-Technologies/ml-agents"

3 A. B. Juliani, "Unity: A General Platform for Intelligent Agents"

4 J. C. H. Watkins, "Q-learning" 8 (8): 272-292, 1992

5 S. Y. Park, "Object-spatial layout-route-based hybrid nap and its application to mobile robot navigation" Graduate School of Yonsei University 2010

6 D. Filliat, "Map-based navigation in mobile robots: I. A review of localization strategies" 4 (4): 243-282, 2003

7 J. A. Meyer, "Map-based navigation in mobile robots : II. A review of map-learning and path-planning strategies" 4 (4): 283-317, 2003

8 N. J. Cho, "Learning, improving, and generalizing motor skills for autonomous robot manipulation : an integration of imitation learning, reinforcement learning, and deep learning" Graduate School of Hanyang University 2020

9 X. B. Peng, "Deeploco : Dynamic locomotion skills using hierarchical deep reinforcement learning" 36 (36): 1-13, 2017

10 A. E. Sallab, "Deep reinforcement learning framework for autonomous driving" 1 (1): 70-76, 2017

11 J.H.Woo, "Collision Avoidance for an Unmanned Surface Vehicle Using Deep Reinforcement Learning" Graduate School of Seoul National University 2018

12 A. Coates, "Apprenticeship learning for helicopter control" 52 (52): 97-105, 2009

13 B.G.Ahn, "An Adaptive Motion Learning Architecture for Mobile Robots" Graduate school of SungKyunKwan University 2006

14 X. Chen, "A Reinforcement Learning Method of Obstacle Avoidance for Industrial Mobile Vehicles in Unknown Environments Using Neural Network" 1 (1): 671-667, 2014

15 R. S. Sutton, "A Bradford Book" MIT Press 2017