Self-Imitation Learning을 이용한 개선된 Deep Q-Network 알고리즘

선우영민(Yung-Min Sunwoo),이원창(Won-Chang Lee) 한국전기전자학회 2021 전기전자학회논문지 Vol.25 No.4

Self-Imitation Learning은 간단한 비활성 정책 actor-critic 알고리즘으로써 에이전트가 과거의 좋은 경험을 활용하여 최적의 정책을 찾을 수 있도록 해준다. 그리고 actor-critic 구조를 갖는 강화학습 알고리즘에 결합되어 다양한 환경들에서 알고리즘의 상당한 개선을 보여주었다. 하지만 Self-Imitation Learning이 강화학습에 큰 도움을 준다고 하더라도 그 적용 분야는 actor-critic architecture를 가지는 강화학습 알고리즘으로 제한되어 있다. 본 논문에서 Self-Imitation Learning의 알고리즘을 가치 기반 강화학습 알고리즘인 DQN에 적용하는 방법을 제안하고, Self-Imitation Learning이 적용된 DQN 알고리즘의 학습을 다양한 환경에서 진행한다. 아울러 그 결과를 기존의 결과와 비교함으로써 Self-Imitation Leaning이 DQN에도 적용될 수 있으며 DQN의 성능을 개선할 수 있음을 보인다. Self-Imitation Learning is a simple off-policy actor-critic algorithm that makes an agent find an optimal policy by using past good experiences. In case that Self-Imitation Learning is combined with reinforcement learning algorithms that have actor-critic architecture, it shows performance improvement in various game environments. However, its applications are limited to reinforcement learning algorithms that have actor-critic architecture. In this paper, we propose a method of applying Self-Imitation Learning to Deep Q-Network which is a value-based deep reinforcement learning algorithm and train it in various game environments. We also show that Self-Imitation Learning can be applied to Deep Q-Network to improve the performance of Deep Q-Network by comparing the proposed algorithm and ordinary Deep Q-Network training results.

A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning

Duk-Sun Shim,Joseph Shim 제어·로봇·시스템학회 2023 International Journal of Control, Automation, and Vol.21 No.11

An optimization algorithm is essential for minimizing loss (or objective) functions in machine learning and deep learning. Optimization algorithms face several challenges, one among which is to determine an appropriate learning rate. Generally, a low learning rate leads to slow convergence whereas a large learning rate causes the loss function to fluctuate around the minimum. As a hyper-parameter, the learning rate is determined in advance before parameter training, which is time-consuming. This paper proposes a modified stochastic gradient descent (mSGD) algorithm that uses a random learning rate. Random numbers are generated for a learning rate at every iteration, and the one that gives the minimum value of the loss function is chosen. The proposed mSGD algorithm can reduce the time required for determining the learning rate. In fact, the k-point mSGD algorithm can be considered as a kind of steepest descent algorithm. In a real experiment using the MNIST dataset of hand-written digits, it is demonstrated that the convergence performance of mSGD algorithm is much better than that of the SGD algorithm and slightly better than that of the AdaGrad and Adam algorithms.

딥러닝 알고리즘 기반기술의 스마트 정보 활성화 방안에 관한 연구

정분도,홍미선 국제e-비즈니스학회 2019 e-비즈니스 연구 Vol.20 No.2

The 4th Industrial Revolution has developed a deep learning algorithm that can enhance the artificial intelligence of computers and for the implementation of the neural network, we classify the data sets by mimicking the connectivity of the human brain and find the correlation between the data. The technology that has the ability to learn on its own is the core of artificial intelligence technology that utilizes the deep learning algorithm and it is the last industrial revolution that combines artificial intelligence and object sensor technology. Manufacturing which has grown from application of fragmentary technology is changing to a new growth paradigm and State-of-the-art technologies are being developed and commercialized. Therefore it is necessary to develop platform that can automatically select an object recognition model and hyper parameters on the deep learning algorithm of the user's desired condition based on statistics. The purpose of this study is to present the analytical foundation of the deep learning algorithm from the practical point of view rather than presenting the technical direction of the deep learning algorithm. 4차 산업혁명은 컴퓨터의 인공지능을 높일 수 있는 딥러닝 알고리즘을 개발하여 사용하고 있으며, 심층신경망 구현을 위해 인간 두뇌의 연결성을 모방하여 데이터 세트를 분류하고 데이터 간의 상관관계를 찾게된다. 딥러닝 알고리즘 활용의 인공지능 기술은 스스로 자기가 학습할 수 있는 능력을 갖추는 기술이 핵심이며, 인공지능과 사물센서 기술이 융합된 마지막 산업혁명이다. 그동안 단편적인 기술 적용으로 성장해왔던 제조업이 새로운 성장 패러다임으로 바뀌고 있으며, 최첨단기술들이 계속 발전되어 상품화 되고 있다. 그러므로 사용자가 원하는 조건의 딥러닝 알고리즘 기반의 사물인식 모델 및 하이퍼 파라미터를 통계에 기반하여 자동으로 선정할 수 있는 플랫폼 등이 개발되어야 한다. 본 연구는 딥러닝 알고리즘의 기술적 방향의 제시보다는 실무적 관점에서 살펴본 후, 향후의 해석적 기초를제시하고자 한다.

보행 역학 데이터를 활용한 대퇴 의지 실시간 제어를 위한 딥러닝 모델 구조 연구

김종운(Jong Un Kim),이현주(Hyun Ju Lee),이영식(Young Sik Lee),함수림(Su Lim Ham),조현석(Hyeon Seok Cho),태기식(Ki Sik Tae) 제어로봇시스템학회 2021 제어·로봇·시스템학회 논문지 Vol.27 No.10

Training data of the deep learning model for the control of the above-knee prosthesis should have a sufficient amount of data to fit with the model. Also, this data can be used for gait classification to control the prosthesis. However, in real-time control, the number of the gait dynamics data counted by the sensor is only measured to the level that the deep learning model is difficult to learn., In this study, the most efficient deep learning model case was developed to resolve this problem in the case of a real-time control situation where data measurement time is insufficient. The data is collected through a hall sensor, load cell and Inertial Measurement Unit (IMU) mounted on the above-knee prosthesis. Subsequently, the collected data is divided into 5 phases (Loading Response (LR), Mid Stance (MS), Push Off (PO), Early Swing (ES), and Late Swing (LS)) of the gait cycle according to the point of inflection of hall sensors and load cells. Afterward, training data of the deep learning were generated by sliding window algorithm and the treated data was exercised on four deep learning models by changing the value of width and depth configurations. The results are assessed on accuracy, loss and F1-Score. In conclusion, the loss function statistically decreased in the case of the values of width and depth of the deep learning model were low. Accuracy and F1-Score were not shown significant difference statistically. Therefore, the results provided an efficient deep learning model for above-knee prosthesis to gait analysis and expected to lead to a more reasonable gait model for the control of the above-knee prosthesis via a more detailed gait classification study in the future.

An Experimental Comparison of CNN-based Deep Learning Algorithms for Recognition of Beauty-related Skin Disease

Chang-Hui Bae(배창희),Won-Young Cho(조원영),Hyeong-Jun Kim(김형준),Ok-Kyoon Ha(하옥균) 한국컴퓨터정보학회 2020 韓國컴퓨터情報學會論文誌 Vol.25 No.12

본 논문에서는 딥러닝 지도학습 알고리즘을 사용한 학습 모델을 대상으로 미용 관련 피부질환 인식의 효과성을 실험적으로 비교한다. 최근 딥러닝 기술을 산업, 교육, 의료 등 다양한 분야에 적용하고 있으며, 의료 분야에서는 중요 피부질환 중 하나인 피부암 식별의 수준을 전문가 수준으로 높인 성과를 보이고 있다. 그러나 아직 피부미용과 관련된 질환에 적용한 사례가 다양하지 못하다. 따라서 딥러닝 기반 이미지 분류에 활용도가 높은 CNN 알고리즘을 비롯하여 ResNet, SE-ResNet을 적용하여 실험적으로 정확도를 비교함으로써 미용 관련 피부질환을 판단하는 효과성을 평가한다. 각 알고리즘을 적용한 학습 모델을 실험한 결과에서 CNN의 경우 평균 71.5%, ResNet은 평균 90.6%, SE-ResNet은 평균 95.3%의 정확도를 보였다. 특히 학습 깊이를 다르게하여 비교한 결과 50개의 계층 구조를 갖는 SE-ResNet-50 모델이 평균 96.2%의 정확도로 미용 관련 피부질환 식별을 위해 가장 효과적인 결과를 보였다. 본 논문의 목적은 피부 미용과 관련된 질환의 판별을 고려하여 효과적인 딥러닝 알고리즘의 학습과 방법을 연구하기 위한 것으로 이를 통해 미용 관련 피부질환 개선을 위한 서비스 개발로 확장할 수 있을 것이다. In this paper, we empirically compare the effectiveness of training models to recognize beauty-related skin disease using supervised deep learning algorithms. Recently, deep learning algorithms are being actively applied for various fields such as industry, education, and medical. For instance, in the medical field, the ability to diagnose cutaneous cancer using deep learning based artificial intelligence has improved to the experts level. However, there are still insufficient cases applied to disease related to skin beauty. This study experimentally compares the effectiveness of identifying beauty-related skin disease by applying deep learning algorithms, considering CNN, ResNet, and SE-ResNet. The experimental results using these training models show that the accuracy of CNN is 71.5% on average, ResNet is 90.6% on average, and SE-ResNet is 95.3% on average. In particular, the SE-ResNet-50 model, which is a SE-ResNet algorithm with 50 hierarchical structures, showed the most effective result for identifying beauty-related skin diseases with an average accuracy of 96.2%. The purpose of this paper is to study effective training and methods of deep learning algorithms in consideration of the identification for beauty-related skin disease. Thus, it will be able to contribute to the development of services used to treat and easy the skin disease.

딥러닝의 모형과 응용사례

안성만(Ahn, SungMahn) 한국지능정보시스템학회 2016 지능정보연구 Vol.22 No.2

Deep learning model is a kind of neural networks that allows multiple hidden layers. There are various deep learning architectures such as convolutional neural networks, deep belief networks and recurrent neural networks. Those have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. Among those architectures, convolutional neural networks and recurrent neural networks are classified as the supervised learning model. And in recent years, those supervised learning models have gained more popularity than unsupervised learning models such as deep belief networks, because supervised learning models have shown fashionable applications in such fields mentioned above. Deep learning models can be trained with backpropagation algorithm. Backpropagation is an abbreviation for “backward propagation of errors” and a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent. The method calculates the gradient of an error function with respect to all the weights in the network. The gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the error function. Convolutional neural networks use a special architecture which is particularly well-adapted to classify images. Using this architecture makes convolutional networks fast to train. This, in turn, helps us train deep, muti-layer networks, which are very good at classifying images. These days, deep convolutional networks are used in most neural networks for image recognition. Convolutional neural networks use three basic ideas: local receptive fields, shared weights, and pooling. By local receptive fields, we mean that each neuron in the first(or any) hidden layer will be connected to a small region of the input(or previous layer’s) neurons. Shared weights mean that we’re going to use the same weights and bias for each of the local receptive field. This means that all the neurons in the hidden layer detect exactly the same feature, just at different locations in the input image. In addition to the convolutional layers just described, convolutional neural networks also contain pooling layers. Pooling layers are usually used immediately after convolutional layers. What the pooling layers do is to simplify the information in the output from the convolutional layer. Recent convolutional network architectures have 10 to 20 hidden layers and billions of connections between units. Training deep learning networks has taken weeks several years ago, but thanks to progress in GPU and algorithm enhancement, training time has reduced to several hours. Neural networks with time-varying behavior are known as recurrent neural networks or RNNs. A recurrent neural network is a class of artificial neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior. Unlike feedforward neural networks, RNNs can use their internal memory to process arbitrary sequences of inputs. Early RNN models turned out to be very difficult to train, harder even than deep feedforward networks. The reason is the unstable gradient problem such as vanishing gradient and exploding gradient. The gradient can get smaller and smaller as it is propagated back through layers. This makes learning in early layers extremely slow. The problem actually gets worse in RNNs, since gradients aren’t just propagated backward through layers, they’re propagated backward through time. If the network runs for a long time, that can make the gradient extremely unstable and hard to learn from. It has been possible to incorporate an idea known as long short-term memory units (LSTMs) into RNNs. LSTMs make it much easier to get good results when traini

Courses Recommendation Algorithm Based On Performance Prediction In E-Learning

Koffi, Dagou Dangui Augustin Sylvain Legrand,Ouattara, Nouho,Mambe, Digrais Moise,Oumtanaga, Souleymane,ADJE, Assohoun International Journal of Computer ScienceNetwork S 2021 International journal of computer science and netw Vol.21 No.2

The effectiveness of recommendation systems depends on the performance of the algorithms with which these systems are designed. The quality of the algorithms themselves depends on the quality of the strategies with which they were designed. These strategies differ from author to author. Thus, designing a good recommendation system means implementing the good strategies. It's in this context that several research works have been proposed on various strategies applied to algorithms to meet the needs of recommendations. Researchers are trying indefinitely to address this objective of seeking the qualities of recommendation algorithms. In this paper, we propose a new algorithm for recommending learning items. Learner performance predictions and collaborative recommendation methods are used as strategies for this algorithm. The proposed performance prediction model is based on convolutional neural networks (CNN). The results of the performance predictions are used by the proposed recommendation algorithm. The results of the predictions obtained show the efficiency of Deep Learning compared to the k-nearest neighbor (k-NN) algorithm. The proposed recommendation algorithm improves the recommendations of the learners' learning items. This algorithm also has the particularity of dissuading learning items in the learner's profile that are deemed inadequate for his or her training.

Dropout Genetic Algorithm Analysis for Deep Learning Generalization Error Minimization

박재균,최은수,강민수,정용규 국제문화기술진흥원 2017 International Journal of Advanced Culture Technolo Vol.5 No.2

Recently, there are many companies that use systems based on artificial intelligence. The accuracy of artificial intelligence depends on the amount of learning data and the appropriate algorithm. However, it is not easy to obtain learning data with a large number of entity. Less data set have large generalization errors due to overfitting. In order to minimize this generalization error, this study proposed DGA(Dropout Genetic Algorithm) which can expect relatively high accuracy even though data with a less data set is applied to machine learning based genetic algorithm to deep learning based dropout. The idea of this paper is to determine the active state of the nodes. Using Gradient about loss function, A new fitness function is defined. Proposed Algorithm DGA is supplementing stochastic inconsistency about Dropout. Also DGA solved problem by the complexity of the fitness function and expression range of the model about Genetic Algorithm As a result of experiments using MNIST data proposed algorithm accuracy is 75.3%. Using only Dropout algorithm accuracy is 41.4%. It is shown that DGA is better than using only dropout.

드론 영상 기반 딥러닝 알고리즘을 이용한 불법 주정차 번호 인식 기술

이근상(Lee, Geun Sang) 한국지적정보학회 2020 한국지적정보학회지 Vol.22 No.3

최근 도시개발에 따른 불법 주정차 문제는 화재나 응급환자 발생시 교통 흐름을 방해하여 막대한 인명 및 재산피해를 가져오고 있다. 본 연구에서는 이러한 문제를 개선하기 위해 드론 영상 기반의 딥러닝 알고리즘을 이용하여 불법 주정차 번호를 인식하는 연구를 수행하였다. 먼저 50,232개의 차량 번호 학습자료를 구축하였으며 Single Shot Multi-Detector 알고리즘을 이용하여 차량 및 번호판 영역을 식별하였다. 또한 데이터 확장 알고리즘을 이용하여 경사지거나 비틀어진 번호판을 정형화시켰으며, 최종적으로 앵커박스 생성 및 딥러닝 기반의 차량번호 인식기술을 개발하였다. 본 연구에서는 불법 주정차 단속 업무를 효과적으로 지원하기 위해 Visual Studio 2017 환경에서 C++와 C# 언어를 이용하여 차량번호를 자동으로 인식할 수 있는 프로그램도 개발하였으며, 자체 테스트한 차량번호 인식 정확도는 99.4%로 매우 높게 나타났다. 불법 주정차 번호 인식을 위해 전주시 6개 노선을 선정하였으며 드론을 통해 해상도별 영상자료를 구축하였다. 딥러닝 알고리즘을 이용하여 차량 인식 정확도를 평가한 결과 불법 주정차된 64대의 차량 중 62대를 인식하여 96.9%의 높은 인식률을 확보할 수 있었다. 다만 전체 훈련자료 중 약 1.6%로 상대적으로 훈련자료가 부족한 세자리 숫자 번호판이 위치한 노선에서는 차량을 인식하지 못하는 한계를 보였으며, 향후 연구에서는 많은 학습자료 구축을 통해 정확도를 향상시킬 계획이다. Recently, the problem of illegal parking and stopping caused by urban development has caused enormous human and property damage by obstructing the traffic flow in case of fire or emergency patients. In this study, in order to improve this problem, a study was conducted to recognize illegal parking car numbers using a deep learning algorithm based on drone images. First, 50,232 vehicle numbers of various types were constructed as learning data, and the vehicle and license plate areas were identified using the Single Shot Multi-Detector algorithm. In addition, we developed a data expansion algorithm that formalizes inclined or twisted license plates for optimal anchor box and deep learning algorithm application. And finally, an anchor box creation and deep learning-based vehicle number recognition technology were developed. In this study, a program that can automatically recognize vehicle numbers using C++ and C# languages in the Visual Studio 2017 environment was also developed to effectively support illegal parking and stopping enforcement work. In addition, the self-tested vehicle number recognition accuracy was very high at 99.4%. For the recognition of illegal parking and stop car numbers, six routes in Jeonju were selected as a representative. And image data by resolution were constructed through drone photography. As a result of analysis through a deep learning algorithm, 62 out of 64 illegally parked and stopped vehicles were recognized, ensuring a high accuracy of 96.9%. However, about 1.6% of the total training data showed a limitation in not being able to recognize vehicles on the route where the three-digit license plate was relatively insufficient. And in future studies, it is necessary to improve the accuracy by securing many learning materials.

Dropout Genetic Algorithm Analysis for Deep Learning Generalization Error Minimization

Park, Jae-Gyun,Choi, Eun-Soo,Kang, Min-Soo,Jung, Yong-Gyu The International Promotion Agency of Culture Tech 2017 International Journal of Advanced Culture Technolo Vol.1 No.1

Recently, there are many companies that use systems based on artificial intelligence. The accuracy of artificial intelligence depends on the amount of learning data and the appropriate algorithm. However, it is not easy to obtain learning data with a large number of entity. Less data set have large generalization errors due to overfitting. In order to minimize this generalization error, this study proposed DGA(Dropout Genetic Algorithm) which can expect relatively high accuracy even though data with a less data set is applied to machine learning based genetic algorithm to deep learning based dropout. The idea of this paper is to determine the active state of the nodes. Using Gradient about loss function, A new fitness function is defined. Proposed Algorithm DGA is supplementing stochastic inconsistency about Dropout. Also DGA solved problem by the complexity of the fitness function and expression range of the model about Genetic Algorithm As a result of experiments using MNIST data proposed algorithm accuracy is 75.3%. Using only Dropout algorithm accuracy is 41.4%. It is shown that DGA is better than using only dropout.