Dropout Genetic Algorithm Analysis for Deep Learning Generalization Error Minimization

다국어 초록 (Multilingual Abstract)

Recently, there are many companies that use systems based on artificial intelligence. The accuracy of artificial intelligence depends on the amount of learning data and the appropriate algorithm. However, it is not easy to obtain learning data with a large number of entity. Less data set have large generalization errors due to overfitting. In order to minimize this generalization error, this study proposed DGA(Dropout Genetic Algorithm) which can expect relatively high accuracy even though data with a less data set is applied to machine learning based genetic algorithm to deep learning based dropout. The idea of this paper is to determine the active state of the nodes. Using Gradient about loss function, A new fitness function is defined. Proposed Algorithm DGA is supplementing stochastic inconsistency about Dropout. Also DGA solved problem by the complexity of the fitness function and expression range of the model about Genetic Algorithm As a result of experiments using MNIST data proposed algorithm accuracy is 75.3%. Using only Dropout algorithm accuracy is 41.4%. It is shown that DGA is better than using only dropout.

참고문헌 (Reference)

1 "http://yann.lecun.com/exdb/mnist/"

2 Lipowski, A., "Roulette-wheel selection via stochastic acceptance" 391 (391): 2193-2196, 2012

3 Marsland, S., "Machine learning: an algorithmic perspective" CRC press 2015

4 Forrest, S., "Genetic algorithms- Principles of natural selection applied to computation" 261 (261): 872-878, 1993

5 D.E. Goldberg, "Genetic Algorithms in search, optimization, and machine learning" Addison-Wesley 1999

6 Kim, Y, J., "Future society change that artificial intelligence technology development will bring" 12 : 52-65, 2016

7 Srivastava, N., "Dropout : a simple way to prevent neural networks from overfitting" 15 (15): 1929-1958, 2014

8 Park, J. S., "Designing Neural Network Using Genetic Algorithm" 4 (4): 2309-2314, 1997

9 Umbarkar, A, J., "Crossover Operators in Genetic Algorithms : a revice" 6 (6): 1083-1092, 2015

10 Mitchell, M., "An introduction to genetic algorithms" MIT press 1998

11 Holland, J. H., "Adaptation in natural and artificial systems : an introductory analysis with applications to biology, control, and artificial intelligence" MIT press 1992

12 Whitley, D., "A genetic algorithm tutorial" 4 (4): 65-85, 1994

13 Elyan, E., "A genetic algorithm approach to optimising random forests applied to class engineered data" 384 : 220-234, 2017