ChatGTP API 활용 인공지능 논･서술형 자동채점 프로그램 개발 실행연구 = Action Research on the Development Program of ChatGPT Automated Scoring Method Program for Essay Evaluation

다국어 초록 (Multilingual Abstract)

Purpose: With the rapid development of AI technology, discussions on applying AI in the field of education are ongoing. This research aims to develop a program utilizing AI for the automatic scoring of essay-type items without definite answers. Method: The research was the participatory action research method, which integrates researchers and practitioners. Participation in OpenAI-sponsored Hackathon competitions allowed for continuous improvement of the program prototype based on feedback from experts. Results: ｢Evaluative AI(v1.0)｣ was developed, capable of generating assessment scores and comments for essay-type answers according to user-defined rubrics. Additionally, ｢Evaluative AI(v2.0)｣ was developed to reflect experts’ comments at the Hackathon competition, creating ‘AI evaluation teachers’ with individual characteristics, and comparing scores with human scorers. Conclusion: Through the research, several possibilities were proposed: the ability of AI to understand evaluation rubrics and score by reflecting them, the ability to design ‘AI evaluation teachers’ similar to school teachers, the high intra-rater reliability of ‘AI evaluation teacher’; and the reduction of errors through cross-evaluating by multiple ‘AI evaluation teachers’. In addition, a couple of suggestions for the future action research were proposed: research that first train AI a large number of essay-type answers before conducting main essay evaluations, research exploring how teachers utilize AI automatic scoring programs in class, and research for developing the program prototype for coding qualitative data.