유형별로 본 한국어 능력 평가의 실제와 과제

최은규 국제한국어교육학회 2006 한국어 교육 Vol.17 No.2

The purpose of this paper is to analyse the present condition of placement and achievement tests which are being given in the Korean educational institutions and to propose a better plan for the two test types on the basis of the result. The placement tests consist of written tests for writing and interviews for speaking. The placement is done by considering the results of both the written test and the interview. The placement tests need to be more specific through 1) examining the relevance to evaluating scope, 2) studying evaluating items and the number of items according to levels, 3) reviewing the reliability of deciding levels and 4) metrically studying the results of tests. The achievement tests target on the evaluation of learners' having learned and achieved through lessons. They include the mid-term and final examinations that are given in the education institutions for Korean and cover listening, speaking, writing and vocabulary and grammar tests. These should be taken into account for the achievement tests: 1) lowering the proportions of tests and introducing learner centered test systems such as the performance assessment, 2) establishing the assessing category on the basis of the curricula and teaching goals of the education institutions, 3) researching speaking and writing test marking, 4) reviewing and developing assessment types and 5) studying the application of computers to assessment. (Seoul National University)

A Comparative Analysis of Inter-rater Reliability in English Essay and Korean-English Translation Tests

박리리,최인철 현대영어교육학회 2016 현대영어교육 Vol.17 No.1

Writing performance assessment is gaining momentum in the field of English education in Korea. Writing performance tests, however, are prone to subjective rating that may have adverse effects on the reliability of scores. Previous research recommended reinforced training and clearer scoring criteria. Rater training, however, is costly and demanding in the case of essay tests which require understanding and application of multiple criteria without much guidance on benchmark responses. This study thus sought to examine translation as an alternative writing task that may induce greater score reliability as well as accurate measurement of writing proficiency. Three raters scored the responses of 20 Korean college students to an English to Korean translation test and an English essay test solely based on the rubrics. The results showed that inter-rater reliability was higher for the translation test than for the essay test. In addition, the agreement between two of the raters reached a statistically significant level for the translation test. The implications of these findings are discussed in relation to the viability of implementing translation tests in secondary schools together with suggestions for further study on ways to strengthen the advantages of both essay and translation tests and make up for their weaknesses.

Investigating the Effects of Test Takers’ Perceptions on their Writing Performance

이영주 한국영어학회 2011 영어학 Vol.11 No.2

This paper explores the effect that test takers’ perceptions have on their writing performance. This study also investigates the extent to which test takers’ perceptions of the writing test contribute to their performance, controlling for general English proficiency. The test instrument employed in this study is a newly-developed process-oriented ESL writing assessment. 100 international students took an institutional ESL test which allowed test takers to reflect, interact with other task takers,and revise the essays. Participants were asked to fill out a survey right after the test and 10 closed-response items concerning test format, length, perceptions of test validity, and anxiety were used as an indicator of their attitudes toward the test. The test takers were divided into 3 groups based on their ‘fair average’ scores on these 10 items obtained through multifaceted Rasch measurement. Results showed that students who were positively disposed to the test received slightly higher scores than students who were not,although different perceptions did not lead to statistically differential performance. The results of ANCOVA by controlling for TOEFL scores as the covariate showed that students who were more favorably disposed towards the test instrument did not turn out to be significantly more able, compared with those who were less favorably disposed. There was no relationship between test takers’ perceptions and their performance. Although positive attitudes toward the test did not lead to a statistically significant level of performance in this study, it is suggested that language testers nonetheless need to be aware that attitudes toward the test can be a significant factor in test performance.

Reliability and Validity of Using Reorganization-Translation Task for L2 Writing Assessment

박리리 새한영어영문학회 2019 새한영어영문학 Vol.61 No.4

This study explored the potentials of a newly-developed translation test as a useful tool in measuring L2 learners’ writing competence. The Reorganization-Translation test is a type of L1 to L2 translation test that is designed to gauge test-takers’ discourse competence in addition to other writing assessment criteria. For the purposes of this study, the test was administered on 95 college students in Korea, and the test responses were marked by four raters. The analyses in this study were centered on the test’s reliability and construct validity, the two qualities of test usefulness that are of great importance in justifying the use of test results. The findings of this research upheld the usefulness of the test as a reliable and valid form of L2 writing assessment and a helpful instrument in teaching L2 learners about coherence or rhetorical organization in writing. The significance of this study was highlighted in relation to its first-of-its-kind attempt to develop and administer a translation test reflecting all of the components of language ability including discourse competence. The implications of such endeavor are discussed together with the study’s limitations and suggestions for a probe into a more appropriate setting for the test.

Reliability and Validity of Using Reorganization-Translation Task for L2 Writing Assessment

Park, Lili(박리리) 새한영어영문학회 2019 새한영어영문학 Vol.61 No.4

This study explored the potentials of a newly-developed translation test as a useful tool in measuring L2 learners’ writing competence. The Reorganization-Translation test is a type of L1 to L2 translation test that is designed to gauge test-takers’ discourse competence in addition to other writing assessment criteria. For the purposes of this study, the test was administered on 95 college students in Korea, and the test responses were marked by four raters. The analyses in this study were centered on the test’s reliability and construct validity, the two qualities of test usefulness that are of great importance in justifying the use of test results. The findings of this research upheld the usefulness of the test as a reliable and valid form of L2 writing assessment and a helpful instrument in teaching L2 learners about coherence or rhetorical organization in writing. The significance of this study was highlighted in relation to its first-of-its-kind attempt to develop and administer a translation test reflecting all of the components of language ability including discourse competence. The implications of such endeavor are discussed together with the study’s limitations and suggestions for a probe into a more appropriate setting for the test.

성취수준과 성취기준에 따른 중학교 3학년 과학과 자극과 반응 단원 지필평가 문항 분석

고희령(Ko, Heeryung) 성신여자대학교 교육문제연구소 2012 교육연구 Vol.53 No.-

The purpose of this study was to analyse written test items of science, stimulus-response, in middle school 9th grade through achievement standards and levels according to 2007 revised science curriculum. A total of 166 written test items from 11 middle schools in Gyeonggi-do was analyzed for this study. The results were as follows; First, there were no written test items which were against achievement standards according to 2007 revised science curriculum. Second, though there were more written test items than achievement standards according to 2007 revised science curriculum in all schools, one more achievement standards were not included in written test items of science in 6 middle schools. Third, the rate of score on high level achievement items to total score on all level achievement items was varied in every school. Therefore written test items must not be concentrated on specific achievement standards and levels to increase content-validity in science written tests. 경기도의 11개 중학교 3학년 과학과 자극과 반응 단원의 지필 평가 문항을 2007개정교육과정에 따른 성취수준과 성취기준에 따라 분석하였다. 총 166개 문항 중 2007개정교육과정 중학교 3학년 과학과 자극과 반응단원의 8개의 성취기준에 벗어난 내용을 다루고 있는 문항은 없는 것으로 나타났다. 자극과 반응 관련 문항의 총 배점에 대한 각각의 성취기준에 해당하는 문항의 배점 비율을 학교 별로 산출한 결과, 모든 학교에서 출제하고자 하는 문항 수가 성취기준보다 많았음에도 불구하고 11개교 중 6개교에서 1개 이상의 성취기준이 평가문항에 포함되지 않은 것으로 나타났다. 자극과 반응 관련 문항의 총 배점에 대한 각각의 성취수준에 해당하는 문항의 배점 비율을 학교 별로 산출한 결과, 상의 성취수준에 해당하는 문항의 배점 비율이 학교에 따라 큰 차이가 있는 것으로 나타났다. 지필평가의 타당도를 높이기 위하여 특정 성취기준과 성취수준을 평가하는 문항들이 편중되지 않도록 하는 것이 필요하다.

Comparing Cross-disciplinary Test Takers’ Performance between Internet-based and Paper-based Diagnostic Writing Tests in EFL College English Classrooms

Li Jiang 중앙대학교 외국학연구소 2021 외국학연구 Vol.- No.57

This study explores differences between EFL students’ performance in Internet-based and Paper-based Diagnostic Writing Tests. A recently developed e-learning app Rain Classroom is used to conduct the Internet-based diagnostic English writing test to 95 college students and an equivalent paper-based test is set for comparison. The students’ scores on these two tests are analyzed through t-test and no statistically significant difference is found between these two test modes, which implies that some new means should be worth trying for Internet-based diagnostic English writing assessment in college. However, to provide more precise suggestions, students in the study are further divided into three groups according to the disciplines to which they belong. With the aid of t-test and one-way ANOVA, it is revealed that students in the discipline of international business perform better in the Internet-based test, whereas students in the discipline of art design display a contrary tendency. Additionally, students from the discipline of urban planning demonstrate no salient performance difference between the two test modes. Therefore, some pedagogical implications are then made for the Internet-based diagnostic assessment for L2 college English writing.

성취수준과 성취기준에 따른 중학교 3학년 과학과 자극과 반응 단원 지필평가 문항 분석

고희령(Ko, Heeryung) 성신여자대학교 교육문제연구소 2012 교육연구 Vol.54 No.-

경기도의 11개 중학교 3학년 과학과 자극과 반응 단원의 지필 평가 문항을 2007개정교육과정에 따른 성취수준과 성취기준에 따라 분석하였다. 총 166개 문항 중 2007개정교육과정 중학교 3학년 과학과 자극과 반응단원의 8개의 성취기준에 벗어난 내용을 다루고 있는 문항은 없는 것으로 나타났다. 자극과 반응 관련 문항의 총 배점에 대한 각각의 성취기준에 해당하는 문항의 배점 비율을 학교 별로 산출한 결과, 모든 학교에서 출제하고자 하는 문항 수가 성취기준보다 많았음에도 불구하고 11개교 중 6개교에서 1개 이상의 성취기준이 평가문항에 포함되지 않은 것으로 나타났다. 자극과 반응 관련 문항의 총 배점에 대한 각각의 성취수준에 해당하는 문항의 배점 비율을 학교 별로 산출한 결과, 상의 성취수준에 해당하는 문항의 배점 비율이 학교에 따라 큰 차이가 있는 것으로 나타났다. 지필평가의 타당도를 높이기 위하여 특정 성취기준과 성취수준을 평가하는 문항들이 편중되지 않도록 하는 것이 필요하다. The purpose of this study was to analyse written test items of science, stimulus-response, in middle school 9th grade through achievement standards and levels according to 2007 revised science curriculum. A total of 166 written test items from 11 middle schools in Gyeonggi-do was analyzed for this study. The results were as follows; First, there were no written test items which were against achievement standards according to 2007 revised science curriculum. Second, though there were more written test items than achievement standards according to 2007 revised science curriculum in all schools, one more achievement standards were not included in written test items of science in 6 middle schools. Third, the rate of score on high level achievement items to total score on all level achievement items was varied in every school. Therefore written test items must not be concentrated on specific achievement standards and levels to increase content-validity in science written tests.

Predictability of the Cloze Test as a Measure of Written Productive Vocabulary

유영숙 현대영어교육학회 2017 현대영어교육 Vol.18 No.4

The present study explores whether cloze test scores under two different scoring methods and English test scores on TOEIC can predict written productive vocabulary of Korean EFL learners. One hundred and fourteen students’ written productive vocabulary in summary writing was analyzed using VocabProfile, which provides four frequency-based vocabulary level lists: the most frequent 1,000 words (K1: function words and content words), second most frequent 1,000 words (K2), academic word lists (AWL), and off the list words (OLW). The study found that the exact-word method led to significant differences in K1, function words, K2, AWL, and OLW, and the acceptable-word method resulted in significant differences in K1, function words, K2, and OLW, indicating only one difference in AWL. On the other hand, the English proficiency test yielded significant differences in K1 and function words alone. These study results suggest that the cloze test under the two different scoring methods did not make a remarkable difference in the predictability of Korean EFL learners’ written productive vocabulary. In addition, both cloze test scores and general English test scores showed they could serve as predictors of Korean EFL learners’ written productive vocabulary, but cloze test scores led to more significant differences in vocabulary use in writing.

고등학교 영어 수행평가와 정기고사의 수능 영어 모의고사 예측

이명순 한국영어평가학회 2021 영어평가 Vol.16 No.2

The present study was designed to examine the prediction of preliminary College Scholastic Ability Tests (CSATs) English by high school regular English reading exam and English performance test. The high school English performance test consists of an English listening test, speaking test, and writing test. Two hundred eighty-two Korean male high school students participated in this study. For this analysis, correlation was conducted among preliminary CSAT English, English regular reading exam and English performance test. In addition, regression was conducted and Preliminary CSAT was the dependent variable. The English performance test and English regular exam were independent variables. The results of this study showed that preliminary CSAT English had a high correlation with regular English reading exam and listening performance test. In addition, preliminary CSAT English was usually predicted by the English reading and writing test. This seems to be because preliminary CSAT English was usually testing the learners’ English reading and listening ability. It would be desirable for CSAT English to measure basic English speaking and writing ability.