한국의 영어평가의 발전과 현황과 과제

권오량(Kwon, Oryang) 한국영어평가학회 2013 영어평가 Vol.8 No.1

This paper reviews and discusses the development and current status of English language assessment in Korea and presents some missions for English testing professionals. This paper first reviews the development of college entrance examinations, noting that they had traditionally focused on vocabulary, grammar, and reading before the introduction of the College Scholastic Ability Test (CSAT), which emphasizes communicative competence and introduced a listening comprehension section. The paper then discusses English language testing in school settings. The national curriculums had not put much weight on testing prior to the 2011 curriculum. As for English language assessment in Korean society, the paper describes the extraordinary fever for English language testing by the general public. Next, several issues surrounding English language testing in Korea are discussed, including the government’s control of the difficulty level and content of the CSAT, the National English Ability Test, and productive skills assessment in schools. Finally, the paper discusses the dilemmas that English language testing professionals are faced with, suggesting that professionals have the mission of educating both the government and society for the improvement of English language assessment.

Exploring the Efficacy of Automated Essay Scoring Systems in the Context of Korean EFL High School

Yubin Jeon 한국영어평가학회 2023 영어평가 Vol.18 No.1

Despite growing attention on the Automated Essay Scoring (AES) systems, it is difficult to find their actual use in Korean EFL high schools. Among various free and easily accessible AES systems available, this study aims to investigate the efficacy of three AES systems in the Korean EFL high school context: Grammarly, MS Editor, and ProWritingAid. To this end, the three AES systems were compared with two human raters in three aspects: scores reported on Korean EFL high school students’ writings, errors detected per error category, and errors detected per student writing. The t-test, ANOVA, and correlation analysis results have led to questions and doubts on the validity of the AES programs and their role as essay evaluators. While AES programs have shown growth, the AES programs are not yet comparable to human raters and teacher feedback. However, the AES programs do show potential as a supplementary tool in the Korean EFL context. Thus, a comparison of the three AES systems was examined in the hopes of finding the optimal choice of AES system. The findings provide information and insights on how AES systems may be implemented in Korean EFL high school classrooms, and further emphasized the importance of teachers’ role in implementing a new tool.

A Corpus-based Analysis of the Linguistic Complexity Levels of the Reading Passages in Three Asian EFL Countries' College Entrance English Exams

엄혜랑 한국영어평가학회 2023 영어평가 Vol.18 No.1

This study investigates the linguistic complexity levels of the reading passages of the foreign language (English) domain of the College Scholastic Aptitude Test (CSAT) in Korea and compares them with those in two other Asian EFL countries. For the study, the reading passages of the Korean, Chinese, and Japanese college entrance English exams from the 2018 to 2020 academic year were collected and analyzed based on the length, propositional content and organizational characteristics by using a variety of corpus linguistic tools. The Kruskal-Wallis test was administered in order to see if there was a statistically significant difference among the tests. Also, when between-group differences were detected, the Mann-Whitney U post hoc test was conducted to further look into significant differences between the CSAT and the other tests. Overall findings suggest that the reading passages of the CSAT English exam maintained the highest difficulty level among the three Asian EFL countries’ college entrance English exams, with cognitively demanding academically-oriented items, highly advanced vocabulary, complex syntactic structure and low readability. Based on the findings, this research is expected to shed light on the intention of incorporating criterion-referenced English assessment into the CSAT as well as provide pedagogical implications for test developers and education policy makers in Korea.

Investigation of Improving Pre-service Teachers’ Competence for Formative Assessment Aligned with Primary English Classroom Activities

Jin-Seok Kim 한국영어평가학회 2022 영어평가 Vol.17 No.1

The purpose of this study is to suggest ways of improving learners’ communicative competence through formative assessment aligned with classroom activities for pre-service teachers. To achieve this, four pre-service teachers’ lesson plans designed for the fifth and sixth graders in the lecture of Primary English Methodology were analyzed in terms of the criteria of analyzing focusing on formative assessment. It was found in the results of analyzing lesson plans that 1) pre-service teachers did not appropriately design learning objectives to promote student motivation, 2) all of the pre-service teachers did not design self-assessment, although only one pre-service teacher designed peer-assessment such as “Let’s check partner’s listening skill!”, and 3) all of them did not give feedback systematically to improve learners’ communicative competence, and carry out formative assessment within 5 minutes in the closing stage of primary English classes. From the findings, it was suggested that 1) pre-service teachers need to design strategies for sharing learning targets and criteria for success based on formative assessment, 2) they need to set up lesson objectives based on formative assessment aligned with primary English classroom activities, 3) they need to prepare assessment checklists or sheets for assessment for learning (AFL), considering key competences and skills, 4) they need to consider the core strategies in designing and implementing formative assessment, and 5) they need to give immediate feedback to the learners considering the results of ongoing assessment and key competences as well as accomplishment standards of the 2015 Revised English National Curriculum.

Measuring Communicative Intentions and Capabilities: Learners' English Proficiency as a Supplement to Willingness to Communicate

이한솔,진성한 한국영어평가학회 2023 영어평가 Vol.18 No.1

In this study, we attempted to measure English learners’ communicative intentions and capabilities (CIC) by adding their English proficiency to their willingness to communicate (WTC) as a predictor of the learners’ English use. In the context of communicative language teaching (CLT), learners’ WTC is considered the key to linking important learner factors such as self-perceived competence, anxiety, and motivation. Given that WTC only accounts for learners’ communicative intentions, we hypothesized that learners’ communicative capabilities (i.e., language proficiency) would be a meaningful addition to measuring overall CIC. To test this hypothesis, we collected a total of 67 samples (N = 17,811) and conducted an analysis using a metaanalytic structural equation modeling (MASEM) approach. The results indicated that the learners’ overall CIC successfully predicted their English use, with a significant path coefficient (β = .525, p = .02). Furthermore, we found that the size of the direct contributions of perceived competence and motivation became non-significantly marginal (ps > .05), indicating that adding learners’ proficiency to the WTC to measure the overall CIC can improve the overall explanatory power of the MASEM model to predict learners’ English use.

Direction of Improving Preservice Teachers' Competence of Designing Primary English Lesson Plan: Focused on Scaffolding Based on Dynamic Assessment

Jin-Seok Kim 한국영어평가학회 2023 영어평가 Vol.18 No.1

The purpose of this study is to present the direction of improving preservice teachers’ competence of designing primary English lesson plans focusing on scaffolding based on dynamic assessment. To achieve the purpose, three lesson plans demonstrated for microteaching in the 2023 Lecture of Primary English Methodology were analyzed in terms of scaffolding based on dynamic assessment in alignment with tasks according to the phases of pre-class, during-lass, and post-class. It was found in the result of analyzing the lesson plans that 1) learners’ zone of proximal development (ZPD) was not established based on dynamic assessment for learners to achieve the learning objectives, 2) macro-level scaffolding based on dynamic assessment was not designed appropriately and systematically for learners to improve their communicative competence in the phases of pre-class and post-class, 3) micro-level scaffolding based on dynamic assessment was not designed for learners to interact with a teacher and to answer questions related to lessons’ content in the phase of during-class, and 4) rating scales aligned with tasks were not established based on a variety of scaffolding for learners to effectively solve the problems or complete their tasks in collaboration with classmates. Based on the results, it was suggested that a preservice teacher needs to check and assess learners’ communicative competence considering a rating scale, which involves scaffolding such as feedback for learners to effectively complete their tasks in collaboration with classmates.

Evaluating Quality in Second Language Performance Tests

Glenn Fulcher 한국영어평가학회 2007 영어평가 Vol.1 No.1

Scoring performance tests involves making judgments about what is valued in language san1ples gathered from a small number of tasks, and summarizing this in a score. We invest the score with a meaning that is interpreted in terms of what a learner can do in the real world. This process assumes that we are able to summarize complex performances in numbers, and that we can generalize that summary to contexts and tasks beyond the sample that generated the numbers. All performance testing contains an implicit validity claim that from scores we can predict to communicative success (or failure) in the real world. The focus of much validity research is concerned with the development of rating scales that allow movement from performance to score, and from score to inference. In this paper we outline some of the major scales that have been used in performance testing, and a number of approaches to scale development. We then identify a range of problems that language testers still face and research questions that remain to be addressed. It is argued that scoring and interpreting performance tests is a much more complex process than is often imagined.

고등학교 1학년 전국연합학력평가 영어 지문의 언어적 특성 및 난이도 분석

이주윤 한국영어평가학회 2023 영어평가 Vol.18 No.1

This study examines the linguistic characteristics and the text difficulty of the National Assessment of Academic Achievement (NAAA). First, the linguistic features of NAAA conducted in June, September, and November were compared to examine the difference among the tests by the scope of the assessment. Second, the linguistic features of NAAA and the reading texts in six high school English textbooks were compared. Lastly, the correlation between the linguistic features of NAAA and the correct answer rate for each question was analyzed. Corpus analysis was conducted by using Coh-Metrix 3.0. The indices were selected for descriptive analysis, lexical diversity, connectives, syntactic complexity, and readability. One-way ANOVA, Kruskal-Wallis H test, independent t-test, Mann-Whitney U test, and correlation analysis were conducted to investigate the Coh-Metrix data. The result revealed the statistical differences among NAAA tests held in June, September, and November regarding the differences in ‘mean number of words of sentences and readability score.’ In addition, discrepancies in linguistic features between the text in NAAA and the English textbooks were found. Lastly, it was found that ‘number of words, mean number of words, CELEX Log frequency for all words, argument overlap, and readability score' had a correlation with the correct answer rate. Implications and the limitations of the study were discussed.

Effects of Choice of Essay Prompts on the Quality of Writing

Hyejung Paik 한국영어평가학회 2008 영어평가 Vol.2 No.1

This study investigates effects of choice of essay prompts on the writing performance of test takers in terms of scores in a college English program at a Korean university. Participants were divided into a choice group (N=20) and a no-choice group (N=20) and only the choice group was provided with two different essay topics in two different essay types: expository/private and argumentative/public. Writings were rated analytically to compare the influence of topic choice on the content and organization of the essay with the one on the vocabulary and grammar. Besides the quantitative investigation, the choice group was asked to answer a questionnaire in order to examine the test takers’ preference as well as the reasons for their selection of prompts. Results revealed that the choice of essay prompts had positive effects on test takers; in particular, the choice group had higher scores on content and organization. Those who had considered the essay type only or considered both essay type and topic familiarity when they had made a choice significantly outperformed the other participants. Out of twenty participants in the choice group, nineteen favored having a topic choice, believing that they write better with familiar topics. The findings indicate that having a topic choice has positive effects both on the test takers’ performance and affective side.

Developing a Framework and Prototype of the Korean National English Ability Test for Secondary School Students

Oryang Kwon,Young-shik Lee,Inn-chull Choi,Dongwan Cho,Youn-hee Choi,Tae-young Jeong,Sang-keun Shin 한국영어평가학회 2008 영어평가 Vol.2 No.1

The present paper is a report on a test development project that developed a framework and prototype of the Korean National English Ability Test for Secondary School Students. The test was a comprehensive test that included all four skills of language: listening, speaking, reading and writing. For the secondary school level, three level tests were developed: S1 for 7th and 8th grades, S2 for 9th and 10th grades, and S3 for 11th and 12th grades. The revised 7th National Curriculum was the basis for the achievement criteria and for the test specifications. Because of the constraints of space, the present paper reports on the development of the listening test and the speaking test only, in terms of test specifications and procedures.