은닉층 특징정보 양자화를 통한 고속 다층 신경회로망 설계 = High speed multilayer neural network design by hiddenlayer feature information quanization

다국어 초록 (Multilingual Abstract)

The multi-layer neural network structure that uses back-propagation algorithm to learning algorithm is often utilized for solving complicated problems of artificial perception such as pattern recognition, computer vision, and phonetic recognition. However, these calculation amounts should design a suitable optimum neural network structure to solve a big problem.
Especially, in the case of multi-layer neural network structure, the decision of the number of hidden layer and hidden node is very important. The hidden node plays a role of the functional units that classifies the features of input pattern in the given question. However, there is a problem that decides the number of hidden nodes based on back-propagation learning algorithm. If the number of hidden nodes is designated very small, perfect learning is not done because the input pattern given cannot be classified enough. On the other hand, if designated a lot, overfitting occurs due to the unnecessary execution of operation and extravagance of memory point. So, the recognition rate is been law and the generality is fallen. Therefore, a neural network that consists of the number of a suitable optimum hidden node has be on the rise as a factor that has an important effect upon a result. The existing neural network structure design process is a field that a fixed principle does not exist, so it has depended entirely upon subjective experiencing knowledge and trial and error of neural network development experts. According to this, various researches were progressed for the optimum neural network structure design, this method decides the number of hidden node using the error sum of spreading information during attending the study. However there is a disadvantage that eliminates an available hidden node because this method only uses output value of hidden layers for pruning hidden node.
The power of neural network is dominated by parameters of learning algorithm, especially, influenced by weights, the number of article of hidden layers, and the number of article of nodes.
Therefore, this monograph suggests a method that decides the number of neural network node with feature information consisted of the parameter of learning algorithm. This method uses for the parameter of learning algorithm that improves weight and offset that were come to the front as a problem of existing back-propagation learning algorithm.
When looks for new weight, the improved weight reflects the changing rate of the error about output value of hidden node used in error function, and although the output value of hidden node changes, the improved weight prevents the error changing.
The improved offset rule can restrain vibrating phenomenon reaching at the global minimum value as reflecting total errors changed by h e in offset, which uses sigmoid function for the threshold function. It can seek for the feature information of hidden layer using the improved weight, offset, and output value of hidden layer, and the feature information is used for the estimated value pruning the hidden node.
It excludes a node in the pruning target, that has a maximum value among the feature value obtained and compares the average of the rest of hidden node feature value with the feature value of each hidden node, and then would like to improve the learning speed of neural network deciding the optimum structure of the multi-layer neural network as pruning the hidden node that has the feature value smaller than the average.

목차 (Table of Contents)

• 목차 = ⅰ
• 표목차 = ⅳ
• 그림목차 = ⅴ
• ABSTRACT = ⅶ
• Ⅰ. 서론 = 1
• Ⅱ. 다층 신경망 구조 = 6
• A. 신경망의 대표적인 활성화 함수 = 7
• B. 다층 퍼셉트론 = 8
• 1. 3계층 MLP 구조 = 9
• 2. MLP 구조의 중요성 = 10
• 3. 다층 신경망 구조 결정요인 = 13
• Ⅲ. 신경망 학습 = 17
• A. Back-Propagation Algorithms = 18
• B. 오류역전파 학습규칙의 문제점 = 21
• Ⅳ. 다층 신경망 구조 결정 방법 = 23
• A. 최적화된 은닉노드를 갖는 NBP 알고리즘 = 23
• 1. 초기설정 = 23
• 2. 은닉노드 결정법 = 24
• 3. 학습곡선과 C_(j)와의 관계 = 25
• B. 유전자 알고리즘을 이용한 은닉노드 제거에 의한 구조설계 = 26
• 1. GA를 이용한 MLP 학습 = 27
• 2. MLP의 구조결정 = 28
• C. 유전자 알고리즘을 이용한 은닉노드 수 결정에 의한 구조 설계 = 30
• 1. 신경망 구조 설계를 위한 구성 = 30
• 2. 다층신경망 구조 결정방법 = 33
• Ⅴ. 은닉층 특징정보를 이용한 은닉 노드 수 결정법 = 36
• A. 다층신경망 구조 결정법의 문제점 = 36
• B. 특징정보 생성을 위한 오류 역전파 알고리즘의 문제점 개선 = 37
• 1. 연결강도 설정 = 37
• 2. offset rule = 41
• C. 특징정보 생성 = 43
• Ⅵ. 시뮬레이션 = 47
• A. XOR = 48
• 1. XOR에서 은닉노드 수 결정과 r.m.s 학습오차 = 48
• 2. 최적화 학습수렴속도 = 52
• B. 전가산기 (Full-Adder) = 54
• 1. 전가산기에서의 은닉노드 수 결정과 r.m.s 학습오차 = 54
• 2. 최적화 학습수렴속도 = 58
• C. 패리티 비트 (Parity bit) = 60
• 1. 4 bit Parity에서의 은닉노드 수 결정과 r.m.s 학습오차 = 61
• 2. 최적화 학습수렴속도 = 64
• D. 오프라인 필기체 영문자 인식 = 66
• 1. 오프라인 필기체 인식을 위한 인식시스템 구성 = 66
• 2. 필기체 영문자 인식에서의 은닉노드 수 결정과 r.m.s 학습오차 = 70
• 3. 최적화 학습수렴 속도 = 72
• 4. 필기체 영문자 인식을 위한 인식률 = 72
• E. 최적화 알고리즘을 적용한 기온예측 = 74
• 1. 신경망을 적용한 기온예측 시스템 구성 = 74
• 2. 기온예측을 위한 은닉노드 수 결정과 r.m.s 학습오차 = 76
• 3. 기상예측을 위한 일반성 = 78
• Ⅶ. 결론 = 80
• 참고문헌 = 84
• 부록. 은닉층 특징정보를 이용한 오류역전파 알고리즘 = 89