온톨로지 지식 기반 특성치를 활용한 Bidirectional LSTM-CRF 모델의 시퀀스 태깅 성능 향상에 관한 연구

다국어 초록 (Multilingual Abstract)

This paper proposes a methodology applying sequence tagging methodology to improve the performance of NER(Named Entity Recognition) used in QA system. In order to retrieve the correct answers stored in the database, it is necessary to switch the user’s query into a language of the database such as SQL(Structured Query Language). Then, the computer can recognize the language of the user. This is the process of identifying the class or data name contained in the database. The method of retrieving the words contained in the query in the existing database and recognizing the object based on the index does not identify the homophone and the word phrases because it does not consider the context of the user’s query. If there are multiple search results, all of them are returned as a result, so there can be many interpretations on the query and the time complexity for the calculation becomes large. In addition, in the existing QA system, chunking has a higher probability of selecting the longest phrase among the sentence component phrases retrieved after obtaining a subset of all cases that can come from the user query. This chunking process has a problem that the algorithm is complicated, but it is not always correct according to the query. To overcome these, this study aims to solve this problem by reflecting the contextual meaning of the query using neural network-based methodology and to identify the problem of the neural network based methodology. Context-sensitive tagging combined with CRF and Bidirectional LSTM can be used to supplement the LSTM-based model, which can limit long-term memory problems, but can be biased toward recent input because of the data nature of word embedding. Therefore, we have solved the disadvantages of the neural network model by introducing the latest technology Bidirectional LSTM-CRF model in Sequence Tagging field. We used reasoning that reflects context by using ontology-based characteristic values for untrained words. In case that untrained words come in, we store the object name recognition tag information obtained from the ontology knowledge base in the Lucene index DB as the ontology knowledge based feature. and In this paper, we propose a neural network model based on ontology knowledge based feature.
Experiments were conducted on the ontology knowledge base of music domain and the performance was evaluated. In order to accurately evaluate the performance of the L-Bidirectional LSTM-CRF proposed in this study, we experimented with converting the words included in the learned query into untrained words in order to test whether the words were included in the database but correctly identified the untrained words. Through experimentation, it was confirmed that tag recognition based on the context of the homophone and chunking is possible. In addition, we could apply the features that match the data type by using the CRF property which can use user defined function as a variable. As a result, it was possible to recognize objects considering the context and can recognize the untrained words without re-training the L-Bidirectional LSTM-CRF model, and it is confirmed that the performance of the object recognition as a whole is improved. Therefore, the proposed L-Bidirectional LSTM-CRF methodology can be applied to the ontology knowledge base of various fields to solve the object name recognition problem. However, the proposed L-Bidirectional LSTM-CRF method performed better than the conventional LSTM-CRF method, but lacked the ability to process homophones compared to the conventional Bidirectional LSTM-CRF. This is due to the fact that the hypothesis of this study inserts a feature that can infer a pattern of unknown words based on the entity name pattern of trained queries. This is because the weight increases linearly with the inclusion in the training data. In future research, it is necessary to experiment with various methods of applying weights so that homophones can be found well while ma

참고문헌 (Reference)

1 ZHANG, Xiang., "Text understanding from scratch"

2 GRAVES, Alex., "Speech recognition with deep recurrent neural networks" IEEE 6645-6649, 2013

3 LAMPLE, Guillaume, "Neural Architectures for Named Entity Recognition" 2016

4 HOCHREITER, Sepp., "Long short-term memory" 9 (9): 1735-1780, 1997

5 GRAVES, Alex., "Framewise phoneme classification with bidirectional LSTM and other neural network architectures" 18 (18): 602-610, 2005

6 ELMAN, Jeffrey L., "Finding structure in time" 14 (14): 179-211, 1990

7 LING, Wang, "Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation" 1520-1530, 2015

8 ZHOU, Jie., "End-to-end learning of semantic role labeling using recurrent neural networks" 1127-1137, 2015

9 MIKOLOV, Tomas, "Efficient estimation of word representations in vector space"

10 LAFFERTY, John., "Conditional random fields: Probabilistic models for segmenting and labeling sequence data" 2001

11 KIM, Yoon, "Character-aware neural language models" 2015

12 HUANG, Zhiheng., "Bidirectional LSTM-CRF Models for Sequence Tagging;Named entity recognition with bidirectional LSTM-CNNs"