http://chineseinput.net/에서 pinyin(병음)방식으로 중국어를 변환할 수 있습니다.
변환된 중국어를 복사하여 사용하시면 됩니다.
- 中文 을 입력하시려면 zhongwen을 입력하시고 space를누르시면됩니다.
- 北京 을 입력하시려면 beijing을 입력하시고 space를 누르시면 됩니다.
RISS 인기검색어
- 검색키워드
- 저자 : 서원화 (검색결과 1 건)
-
서원화 Graduate School, Korea University 2018 국내박사
ABSTRACT Validation of the Criterion-referenced English Assessment in the College Scholastic Ability Test (CSAT) in Korea Wonhwa Seo This study is aimed at validating the English domain of the newly implemented Criterion-referenced College Scholastic Ability Test (CR-CSAT). As the English section of CSAT greatly influences on test-takers, English classrooms, and general English education in Korea, validation of the CSAT is required for successful implementation of the CR-CSAT. This study investigated the new criterion-referenced English test in the CSAT (CR-CSAT) in terms of the framework of test usefulness by Bachman and Palmer (1996). Usefulness consisted of six qualities: reliability, construct validity, authenticity, interactiveness, impact, and practicality. The study first explained the specifications of the test, including test environment, format, and language input, and evaluated its usefulness regarding five tests qualities of the CR-CSAT: reliability, construct validity, authenticity, interactiveness, and the impact on test-takers, teachers, and education. To evaluate construct validity of the CSAT, CSAT samples from 2014 to 2018 were selected and examined. Regarding lexical input over time, there were fluctuations observed in tokens, types, and type token ratio of all sections examined. Such fluctuations and inconsistencies have an adverse effect on the exam fairness and reliability as they fail to consider that the same test-taker who did well in one test could face failure in the next. The task types employed throughout the exams are very limited: both tests include multiple-choice questions which may enhance objectivity and reliability but fail to measure ‘true communicative skills.’ The language input of listening subtest was composed of mainly low level vocabulary while the vocabulary of reading subtest was high. To examine reliability of the CR-CSAT, the researcher conducted a number of analyses. Grade level and test score data from 1,226 students who participated in the simulated CSAT in June, 1,218 students who participated in the simulated CSAT administered in September, and 1,195 students who took the 2018 CSAT were analyzed. Analyses included qualitative analyses of test content and quality, a correlation analysis of the test scores, and an item analysis based on CTT and IRT collected from the two mock CSATs and the 2018 CSAT. Results of this study based on CTT and IRT analyses revealed that the difficulty and discrimination of the listening subtest items were very low whereas some reading test items were extremely difficult but their discrimination power was very low. These items need revision. Items with the low degree of difficulty and discrimination were allocated three points while some difficult items were assigned two points. As a result of this problematic score scaling, test-takers within a higher ability parameter obtained a lower scaled score than those within a lower ability parameter. It is desirable to revise test items which are extremely easy or difficult and inappropriately discriminate in order to design more reliable test items. Three native English-speaking instructors were invited to examine the test items of the CSATs from 2014 to 2018. The results revealed that although the CSAT had relatively high internal reliability, the consistency between the tests of different years and the test discriminability did not reach a satisfactory level. Some test items that require revision were found through both quantitative and qualitative item analyses. For the evaluation of the authenticity and interactiveness analysis, six raters were asked to score the degree of authenticity and interactiveness of test item types of the 2018 CSAT and indirect speaking type was found to have the lowest degree of authenticity and interactiveness. On the other hand, the item types of understanding details and understanding context in both listening and reading comprehension displayed high degrees of authenticity and interactiveness. Overall the reading subtests’ authenticity and interactiveness were higher than those of the listening subtest. Raters’ qualitative review supported the results of quantitative analysis. Concerning the representation of national curriculum, the subject matters of the listening subtest were confined to general and familiar topics whereas subject matters of the reading subtest were concentrated on specified academic topics such as social studies or natural sciences. A diversification of topics needs to be considered when designing the CSAT. Finally, a survey of teachers’ and students’ perceptions toward the 2018 CR-CSAT was conducted using a questionnaire survey collected from 372 high school students and 102 secondary school teachers. A descriptive analysis of the questionnaire results was completed. It indicated that the CSAT has substantial impacts on English teaching and learning in high schools. The survey results revealed that high school students and English teachers’ perception of the CSAT in terms of test usefulness: reliability, construct validity, authenticity, interactiveness, and impact. One noticeable result was that the two groups both did not think that the 2018 CSAT contributed the equality of English education, decreased private education cost, or changed language teaching and learning. From the qualitative analysis of the high school English teachers’ responses, the number of students who applied to English in after-school program reduced significantly after the implementation of the 2018 CSAT. Also, the number of English class hours decreased. English teachers who participated in the written interview provided alternatives for successful implementation of the CR-CSAT and normalization of the public English education. The findings from this study provide opportunities to explore pedagogical implications for entrance test developers and education policy makers in Korea in regards to exam quality and methods. The CSAT should be developed by including test items more related to the current national English curriculum and by fulfilling the need for construct validity, authenticity, and positive impacts on both candidates and society. It is important that the various voices of the stakeholders are known to the public in order to promote further discussion on how to improve test quality and to normalize public English education. This study will contribute to validate the first implemented 2018 CR-CSAT based on the Bachman & Palmer’s test usefulness framework. Finally, it is hoped that the study’s comprehensive investigation of the test qualities of construct validity, reliability, authenticity, interactiveness, and impact by analyzing quantitative and quantitative data will raise the awareness of different stakeholders of the CSAT.
연관 검색어 추천
이 검색어로 많이 본 자료
활용도 높은 자료
