트랜스포머 기반 모델을 이용한 연구논문 멀티레이블 주제영역 분류 = Multi-label Topic Classification of Research Papers Using Transformer-based Models

다국어 초록 (Multilingual Abstract)

Purpose To develop MLP and transformer-based models for the multi-label topic classification of research papers using abstract text.
Methods Abstracts from 119,600 papers in the Computer Science category of arXiv were collected to create a multi-label dataset with up to three categories out of a total of 15 possible categories. Performance was evaluated by developing a baseline MLP model along with transformer-based models: BERT, RoBERTa, and DistillBERT.
Results The transformer models outperformed the traditional MLP model. The DistillBERT model achieved the highest micro F1-score of 0.749, while the BERT model recorded macro and weighted F1-scores of 0.655 and 0.733, respectively. The RoBERTa model excelled in the samples method with a score of 0.772.
Conclusion This study enables researchers to quickly explore recent findings and effectively identify their research topics. Additionally, it is expected to significantly contribute to the efficient sharing of academic knowledge and the revitalization of the research community.

참고문헌 (Reference)

1 박자현 ; 송민, "토픽모델링을 활용한 국내 문헌정보학 연구동향 분석" 30 (30): 7-32, 2013

2 박주섭 ; 홍순구 ; 김종원, "토픽모델링을 활용한 과학기술동향 및 예측에 관한 연구" 22 (22): 19-28, 2017

3 권찬양 ; 양현모, "텍스트마이닝을 이용한 한국응급구조학회지 중심단어 분석" 24 (24): 85-92, 2020

4 김성현 ; 김영민, "앙상블 알고리즘과 BERT를 이용한 연구논문 주제영역 분류" 29 (29): 19-33, 2024

5 김상백, "비지도학습 기반 자동 특허문서 분류 시스템" 29 (29): 421-422, 2021

6 김성현 ; 옥창훈 ; 김영민, "머신러닝 기반 인공지능 특허 품질 예측" 31 (31): 61-82, 2023

7 김인후 ; 김성희, "딥러닝 기반의 BERT 모델을 활용한 학술 문헌 자동분류" 39 (39): 293-310, 2022

8 Miric, M., "Using Supervised Machine Learning for Large-scale Classification in Management Research: The Case for Identifying Artificial Intelligence Patents" 44 (44): 491-519, 2023

9 Kaushik, V., "Towards a precise understanding of social entrepreneurship : An integrated bibliometric–machine learning based review and research agenda" 191 : 2023

10 고강욱 ; 옥창훈 ; 김영민, "Supply Chain에서 인공지능의 활용에 관한 연구 : 머신러닝 기반의 분류 기법 활용" 28 (28): 123-135, 2023

11 Chowdhury, S., "Research Paper Classification using Supervised Machine Learning Techniques" 1-6, 2020

12 Herrera, F., "Multilabel Classification Problem Analysis, Metrics and Techniques" Springer International Publishing 2016

13 Xia, Y., "Multi-label classification with weighted classifier selection and stacked ensemble" 557 : 421-442, 2021

14 Kim, DH, "Multi-co-training for document classification using various document representations: TF–IDF, LDA, and Doc2Vec" 477 : 15-29, 2019

15 Nam, J., "Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2014, Proceedings, Part II 14" 437-452, 2014

16 Amin, S., "MLT-DFKI at CLEF eHealth 2019: Multi-label Classification of ICD-10 Codes with BERT" 1-15, 2019

17 Zeidi, F., "LegalTurk Optimized BERT for Multi-Label Text Classification and NER"

18 Blei, D. M., "Latent Dirichlet Allocation" 3 : 993-1022, 2003

19 Kandimalla, B., "Large Scale Subject Category Classificationof Scholarly Papers With Deep Attentive Neural Networks" 5 : 600382-, 2021

20 Zhu, X., "Dynamic ensemble learning for multi-label classification" 623 : 94-111, 2023

21 Bogatinovski, J., "Comprehensive comparative study of multi-label classification methods" 203 : 117215-, 2022

22 González-Carvajal, S., "Comparing BERT against traditional machine learning text classification"

23 Garrido-Merchan, E. C., "Comparing BERT Against Traditional Machine Learning Models in Text Classification" 2 (2): 352-356, 2023

24 Wang, R., "Bayesian network based label correlation analysis for multi-label classifier chain" 554 : 256-275, 2021

25 Devlin, J., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" 2019

26 Hafeez, A., "Addressing imbalance problem for multi label classification of scholarly articles" 11 : 74500-74516, 2023

27 Zhang, M. L., "A review on multi-label learning algorithms" 26 (26): 1819-1837, 2013

28 Tarekegn, A. N., "A review of methods for imbalanced multi-label classification" 118 : 107965-, 2021

29 Sorower, M. S., "A literature survey on algorithms for multi-label learning" Oregon State University 2010