언어능력 평가로서의 말하기 평가 내용 연구 - 한국어 말하기 평가 개발을 위한 국내외 말하기 평가의 비교 분석을 중심으로

원미진 ( Won Mi-jin ),황지유 ( Hwang Ji-you ) 한국문학언어학회(구 경북어문학회) 2018 어문론총 Vol.75 No.-

본고는 언어 능력 평가로서 말하기 평가가 갖춰야 하는 요소를 파악하기 위해 선행하는 외국어 능력시험의 말하기 평가를 검토하고, 이를 바탕으로 하여 한국어능력시험 말하기 평가 개발을 위한 시사점을 제시한 연구이다. 본고에서는 영어, 중국어, 일본어 말하기 평가에서 제시되고 있는 평가 방법의 외적인 구성 형식과 내적인 측정 구인 및 채점 방법을 비교 분석하였다. 구체적으로는 시험 시간과 평가 문항의 유형을 검토하였고, 채점의 구인과 채점 척도 및 방법을 분석하여 이를 바탕으로 한국어 말하기 평가 개발에 있어 고려해야 하는 내용을 제시하였다. 이러한 검토는 그간의 연구들이 이론적인 논의와 교실 현장에서의 말하기 능력에 초점을 맞춘 연구들이었다는 점에 비추어 실용적이고 타당도 높은 말하기 평가 개발을 위해 검토가 필요하다는 전제에서 이루어졌다. 한국어능력시험에서 말하기 평가가 개발될 때 갖춰야 하는 평가로서의 내적, 외적 타당도를 갖추기 위해서는 다른 외국어 시험의 말하기 평가와 비교하여 크게 다르지 않은 범위에서 한국어능력시험의 목적과 한국어의 언어 체계, 그리고 실제 활용가능한 실용성의 측면이 함께 고려되어야 한다. 본고의 결과는 다양한 말하기 평가의 검토를 통해 한국어 말하기 평가 개발에서 고려되어야 하는 점을 제시하고 있다는 점에 있어 평가 개발을 위한 기초 연구로서 의의가 있을 것이다. This study aims to examine various foreign language competence tests to guide the development of a Korean speaking test. Reviewing several other language competence tests is necessary to determine the validity of a speaking test. In this study, many speaking proficiency tests are analyzed in two parts: external forms and internal contents. External forms include the length of the test and the types of questions, and internal contents include the constructs of the test and scoring. English, Chinese, and Japanese speaking tests at Korea and abroad were analyzed. Based on the analysis results, this study presented some points to consider when developing the Korean speaking test. These points are considered necessary for the development of a practical and valid Korean speaking test. This is because the Korean speaking test should be developed similarly and be comparable to other foreign language speaking tests in order to have validity, and this development should also consider the Korean language system and the test’s practicality. This study suggests several points to be considered in the development of a Korean speaking test by reviewing various speaking tests. The results of this study will be valuable for basic research in Korean speaking test development.

Test-takers’ Performance and Perceptions in Total-Immersive Virtual Reality Paired Speaking Tests

이아름 한국멀티미디어언어교육학회 2023 멀티미디어 언어교육 Vol.26 No.3

This study investigated university students’ performance and perceptions of taking paired English speaking tests in fully immersive virtual reality. A total of 36 college students at an advanced level were recruited to participate in either face-to-face (N = 20) or fully immersive VR speaking tests (N = 16), designed to explore test-takers’ speaking abilities. The students were asked to perform two decision-making tasks and fill out a questionnaire asking about their preference for the test mode and perceived levels of presence after completing the tasks. The study revealed no statistically significant difference in the students’ speaking abilities between the two testing modes, suggesting that the VR paired speaking test format could potentially serve as an alternative to the face-to-face paired speaking test. In relation to the analysis of questionnaire items, the students demonstrated a high level of immersion in the virtual environments. Despite their preference for the face-to-face testing mode, they reported that the VR test could potentially replace the face-to-face speaking test. The semi-structured interview data gathered from three participants further revealed that the students, however, reported some discomfort with the paired speaking test itself due to pairing up with the other person. Furthermore, this study presents the limitations of the study and suggests implications for speaking assessments and future studies.

OPIc-based Speaking Test for English Classes in Korea

Ji-Eun Kim 한국영어어문교육학회 2011 영어어문교육 Vol.17 No.4

The purpose of the current study is to analyze a model of OPIc-based speaking test for Korea English classroom. The research questions were (1). What is the English teachers’ opinion on the current speaking teaching and testing? (2). Is the OPIc speaking test appropriate in the Korea English class? and (3). What are the English education students' perceptions of the OPIc-based speaking test for English classes? To answer the first research question, a survey was conducted targeting one hundred and forty-seven teachers. The result of the survey shows that Korea English teachers’ great concern on the English teaching and speaking test are “communicative competence,” “scoring and criteria,” and “condition and need.” To answer the second research question, the OPIc speaking test for the class was analyzed and it was found that in terms of the communicative competence,” “scoring and criteria,” and “condition and need” issues, OPIc based speaking test is appropriate in Korea English class. For the third research question, Q methodology was used targeting English education students and the result shows that most English education students strongly agree to the statements that the OPIc-based speaking test has the advantage to improve communicative competence. This study may provide the information about creating and adopting tests for English classes and relating standardized test and the English classes.

학문 목적 한국어 말하기 평가 도구 개발의 쟁점과 과제

민병곤,이성준 서울대학교 국어교육연구소 2016 국어교육연구 Vol.38 No.-

This paper examines recent approaches to developing speaking tests and in particular the testing of Korean speaking for academic purposes. The first approach is concerned with testing theory of the language for specific purposes (LSP) and Korean for academic purposes (KAP) as a means of verifying the claims for speaking test development. The second approach is analyzing the validation procedures of TOEFL and IELTS as representative academic speaking tests and relating them to the development of Korean speaking tests for academic purposes. The article argues that the development of Korean speaking tests for academic purposes is founded on the conceptual framework of speaking tests, construct setting, item writing, connecting testing method and validation, scoring rubric and rating scale development, and consideration of educational washback and impact. From this view point, developing Korean speaking tests for academic purposes can be justified in terms of practical needs analysis, construct validity projects, circumstantial specification, and rater effect analysis, and it is suggested that continuous concurrent and predictive validation research be conducted to improve speaking tests. 본 연구는 학문 목적 한국어 말하기 평가의 도구의 개발과 관련한 주요 쟁점과 과제를 제시하는 것을 목적으로 한다. 이를 위해 먼저 학문 목적 말하기 평가를 구성하는 이론적 배경으로서 특수 목적 언어 평가 연구와 학문 목적 한국어 말하기 교육, 그리고 학문 목적 한국어 말하기 평가에 관한 연구들을 검토하였다. 그리고 대표적인 국외 학문 목적 말하기 시험을 실시하고 있는 TOEFL과 IELTS에서 말하기 시험이 개발되어 현재의 모습으로 사용되기까지의 과정을 살펴보고, 이를 통해 학문 목적 한국어 말하기 평가 도구 개발에 관한 시사점을 도출하였다. 이상의 논의를 바탕으로 학문 목적 한국어 말하기 평가 도구 개발의 쟁점으로 제시한 것은 평가의 개념 정립과 구인 설정, 평가 과제 개발, 평가 방법 선택 및 타당화, 채점과 척도 개발, 교육적 환류와 영향력이다. 끝으로 학문 목적 한국어 말하기 평가 도구의 개발을 위한 앞으로의 과제로서 개발 과정에서 응시 목적을 고려하여 요구 분석을 실시하고, 구인타당도 확보와 면밀한 평가 계획서 작성, 그리고 채점자 분석 연구의 필요성을 강조하고, 예측타당도와 공인타당도 연구를 통해 학문 목적 한국어 말하기 평가 도구의 안정화와 품질 개선 작업이 지속적으로 이루어져야 함을 제안하였다.

Korean Test Takers’ TOEIC-Speaking and OPIc Test Preparation

김나희 한국응용언어학회 2016 응용 언어학 Vol.32 No.3

High stakes English speaking tests have been used for evaluating applicants’ language proficiency in academic contexts or in the work place. In these situations, most Korean college students prepare for tests such as TOEIC Speaking or OPIc test in order to get better jobs. Although high-stakes tests pressure students and teachers to raise scores, there is little research about how test takers prepare for English speaking tests in cram schools. One aim of this study was to examine and compare TOEIC Speaking and OPIc test preparation activities in a cram school. Also, I aimed to investigate why and how test takers choose test preparation, the characteristics of test preparation, and how they are influenced by test preparation from the test-takers’ perspectives. Various types of data were collected through observation, interview and documentation. The results indicated that activities in the cram school mode were not significantly influenced by test type. Thus, test takers’ activities may be influenced by teacher’s instruction. The study raises awareness of the importance of instruction by a teacher for appropriate test preparation, and its impact on a test taker in the field of language teaching and testing.

Korean Test Takers" TOEIC-Speaking and OPIc Test Preparation

Nahee Kim 한국응용언어학회 2016 응용 언어학 Vol.32 No.3

High stakes English speaking tests have been used for evaluating applicants’ language proficiency in academic contexts or in the work place. In these situations, most Korean college students prepare for tests such as TOEIC Speaking or OPIc test in order to get better jobs. Although high-stakes tests pressure students and teachers to raise scores, there is little research about how test takers prepare for English speaking tests in cram schools. One aim of this study was to examine and compare TOEIC Speaking and OPIc test preparation activities in a cram school. Also, I aimed to investigate why and how test takers choose test preparation, the characteristics of test preparation, and how they are influenced by test preparation from the test-takers’ perspectives. Various types of data were collected through observation, interview and documentation. The results indicated that activities in the cram school mode were not significantly influenced by test type. Thus, test takers’ activities may be influenced by teacher’s instruction. The study raises awareness of the importance of instruction by a teacher for appropriate test preparation, and its impact on a test taker in the field of language teaching and testing.

한국어 말하기 평가 개발 모형 및 시험타당도 검사 절차

김정태 한국언어문화교육학회 2008 언어와 문화 Vol.4 No.3

This study aims to introduce a test development model to develop a Korean speaking test and validation procedure on it. There is a little published work on Korean speaking test development and validation on them. Therefore, this study attempts to initiate a discussion about it. To measure the Korean speaking proficiency of foreign learners of Korean, language testers need to consider a appropriate development model as a initial step in validation procedure because validation process on a test includes validity evidence gathering from the beginning of test development to the end (Fulcher, 2005; Kim, 2006). Based on the test development models of Bachman and Palmer(1996), Davidson and Lynch (2002), and Kim (2006), the study shows the feasibility of adopting the ADDIE model from the field of instructional technology into Korean language test development. ADDIE stands for (need) analysis, design, development, implementation, and evaluation. In each phase, the study describes features to consider for the development of a Korean speaking test. In the needs analysis phase, language testers collect information on Korean language tests by reverse-engineering test items, where reverse-engineering means test specification writing for a test. In the design and development phases, language testers plan test development procedures, and design and develop test items, test sets, and its administration. In the implementation phase, a test is administered as pilot or a main test. Finally, in the evaluation phase, validation is carried out by using all the information gathered from the previous phases. The information includes validity arguments, test results and their quantitative analysis, and rater training procedures. The test development process is not linear, but also iterative and cyclical. (Pai Chai University)

한국어 말하기 시험의 구인과 과업에 관한 연구 - 교실 평가를 중심으로

지현숙 ( Jee¸ Hyunsuk ),이혜란 ( Lee¸ Hyeran ) 한국외국어대학교 외국어교육연구소 2021 외국어교육연구 Vol.35 No.3

교실 말하기 평가는, 특정한 기간 동안 말하기 능력을 숙달하기 위하여 어떻게 교수-학습했는가와 어떠한 말하기 능력을 성취하기 위해 노력했는가가 구체적으로 담기기 때문에 중요하며, 이후의 한국어 교수-학습에 강한 영향력을 끼치기 때문에 논의할 가치가 크다. 이 논문은 그간 외국어교육에서 이루어진 말하기 평가 연구와 대규모 말하기 시험 체계를 검토한 후 한국어 말하기 시험의 ‘구인 및 과업’을 중심으로 논쟁점을 추출하고 기술하였다. 또한, 팬데믹 시기 동안 한국어 교실의 말하기 평가는 어떻게 수행되고 있는지와 한국어 교수자들은 말하기 교수-학습과 평가에 있어서 어떠한 문제 상황에 처해있는지도 조사하여 해결책을 제안하였다. 구인과 과업을 중심으로 지금까지 수행되어온 한국어 말하기 평가를 검토한 결과는 다음과 같이 요약된다: 첫째, 시험 개발자들은 말하기 시험의 과업 난이도를 다소 혼란스러워하고 있다. 둘째, 학문 목적 학습자 대상의 시험 과업은 난이도도 조절되어야 하지만 ‘실제성’의 부면에서도 비판의 여지가 있다. 셋째, 말하기 능력을 평가하기 위한 구인 간의 위계화가 성글고 연관성이 부족하다. 이 연구의 일차적인 의의는 지금까지 다소 미진한 한국어 교실 평가 연구에 일조한 것에서 찾을 수 있다. 또한, 한국어 교수자가 겪고 있는 팬데믹 시대의 교육 환경적 어려움과 말하기 평가 수행의 난점들을 기술하고 해결 방향을 제안함으로써 한국어 말하기 평가의 질적 향상에 기여한 의의가 있다. This thesis extracted and described controversial points focusing on the 'constructions and tasks' of the Korean speaking test after reviewing the speaking assessment research and large-scale speaking test system that have been performed in foreign language education. In addition, during the pandemic, we investigated how the speaking assessment of the Korean language classroom is being performed and what kind of problems Korean teachers face in speaking teaching-learning and assessment, and suggested solutions. The results of reviewing the Korean speaking assessment performed so far focusing on constructs and tasks are summarized as follows: First, the task difficulty of the speaking test is confusing by the test developer. Second, the difficulty of the test task for academic learners should be adjusted, there is also room for criticism in terms of ‘authenticity’. Third, the hierarchy among constructs for assessing speaking ability is sparse and there is a lack of correlation. The primary significance of this study can be found in its contribution to the assessment of the Korean language classroom, which has so far been rather inexperienced. In addition, it is meaningful that it contributed to improving the quality of Korean speaking assessment by describing the educational environmental difficulties experienced by Korean language teachers in the pandemic era and difficulties in performing speaking assessment and suggesting solutions.

The Effects of Paired Partners’ Proficiency Levels on Test-Takers’ Speaking Test Performance

현일선 이화여자대학교 교과교육연구소 2017 교과교육학연구 Vol.21 No.2

This study attempted to explore the effects of partners' proficiency levels on test scores in paired English speaking test. Although the influence of interlocutor's effects on speaking test performance has been extensively researched in the previous studies, the findings have been inconsistent. The present study explored the effect of partners' proficiency levels on test scores in a paired speaking test. Participants included a total of 107 female adults and college students. Based on their standardized speaking test scores, they were grouped into five levels: Superior, High, Intermediate, Novice-high, and Novice proficiency level. Then, 90 test takers completed the test three times with being paired with three different partners. The partners' proficiency levels were higher, the same as, or lower than their own. The results showed an increase in test scores, as the proficiency level of the partners is higher. Furthermore, a statistically significant differences were found in the test scores of test takers. In light of such effects, these findings indicate that the validity of paired speaking test needs to be reconsidered its use and to find a better way to utilize it in a larger context.

자동채점 기반 영어 말하기 시험 현황 분석

이진화,최윤덕,성민창,김혜영 한국영어교육학회 2023 ENGLISH TEACHING(영어교육) Vol.78 No.2

This study investigated the feasibility of adopting an automatic scoring system (ASS) in a domestic English-speaking education context. Scope, test items, assessment criteria, scoring methods, and reporting strategies of six overseas English-speaking tests utilizing ASSs were examined. Moreover, a comparative analysis was conducted to identify disparities between ASS-based and non-ASS-based speaking tests. Findings were: 1) some ASS-based tests utilized ASS technology throughout the assessment, while others adopted a hybrid scoring system involving human raters; 2) compared to non-ASS-based tests, ASS-based tests used more test items targeting low-level skills such as sound and forms but fewer test items targeting conversation and discourse level skills; 3) pronunciation, fluency, and vocabulary were widely employed as evaluation criteria with sparse use of organization, content, and task completion in most ASS-based tests; 4) differences were minimal in assessment criteria application and score calculation between ASS-based and non-ASS-based tests; and 5) some ASS-based tests provided criteria-specific results and feedback with total scores and proficiency levels.