Hyper Parameter Tuning Method based on Sampling for Optimal LSTM Model = 최적 LSTM 모델 결정을 위한 표본 추출 기반 하이퍼 파라미터 튜닝 방법

다국어 초록 (Multilingual Abstract)

As the performance of computers increases, the use of deep learning, which has faced technical limitations in the past, is becoming more diverse. In many fields, deep learning has contributed to the creation of added value and used on the bases of more data as the application become more divers. The process for obtaining a better performance model will require a longer time than before, and therefore it will be necessary to find an optimal model that shows the best performance more quickly. In the artificial neural network modeling a tuning process that changes various elements of the neural network model is used to improve the model performance. Except Gride Search and Manual Search, which are widely used as tuning methods, most methodologies have been developed focusing on heuristic algorithms. The heuristic algorithm can get the results in a short time, but the results are likely to be the local optimal solution. Obtaining a global optimal solution eliminates the possibility of a local optimal solution. Although the Brute Force Method is commonly used to find the global optimal solution, it is not applicable because of an infinite number of hyper parameter combinations. In this paper, we use a statistical technique to reduce the number of possible cases, so that we can find the global optimal solution.

목차 (Table of Contents)

• Abstract
• I. Introduction
• II. Preliminaries
• III. The Proposed Scheme
• IV. Conclusions
• REFERENCES

참고문헌 (Reference)

1 C. Paar, "Understanding cryptography : a textbook for students and practitioners" Springer Science & Business Media 7-, 2009

2 H. K. Lam, "Tuning of the structure and parameters of neural network using an improved genetic algorithm" 25-30, 2001

3 A. Graves, "Supervised sequence labelling with recurrent neural networks" Springer 35-42, 2012

4 R. Socher, "Recursive deep models for semantic compositionality over a sentiment treebank" 1631-1642, 2013

5 T. Mikolov, "Recurrent neural network based language model" 2010

6 J. Bergstra, "Random search for hyper-parameter optimization" 13 : 281-305, 2012

7 R. V. Hogg, "Probability and statistical inference" Pearson Educational International 204-205, 2015

8 J. Snoek, "Practical bayesian optimization of machine learning algorithms" 2951-2959, 2012

9 B. Qolomany, "Parameters optimization of deep learning models using particle swarm optimization" 1285-1290, 2017

10 S. Hochreiter, "Long short-term memory" 9 (9): 1735-1780, 1997

11 R. E. Korf, "Depth-first iterative-deepening: An optimal admissible tree search" 27 (27): 97-109, 1985

12 L. Deng, "Deep learning : methods and applications" 7 (7): 197-387, 2014

13 T. Pukkala, "A heuristic optimization method for forest planning and decision making" 8 (8): 560-570, 1993

14 C. Zhang, "A compound structure of ELM based on feature selection and parameter optimization using hybrid backtracking search algorithm for wind speed forecasting" 143 : 360-376, 2017

15 Y. Xia, "A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring" 78 : 225-241, 2017

16 A. Tharwat, "A ba-based algorithm for parameter optimization of support vector machine" 93 : 13-22, 2017