The Effectiveness of Automated Essay Scoring in an EFL College Classroom

다국어 초록 (Multilingual Abstract)

This study explores two English classes taught in English at a four-year university to investigate theeffectiveness of using the Automated Essay Scoring (AES) system on the improvement of essay writingabilities of both high level and low level EFL students. The instruments involved in this study includestudents’ essays, Criterion, an automated essay scoring system, and a survey. In order to examine theinfluence of the feedback provided by the Criterion system on the participants’ essays, their first and seconddraft were each evaluated and given holistic scores by the teacher-researcher. Following the evaluation, thescores on the first draft and the revised draft were compared. According to the statistical results, both highand low level students all showed overall improvement in their writing after using Criterion. Following anexamination of their writings, a questionnaire was used to measure the strengths and weaknesses of usingCriterion. Additionally, the students’ perception towards face-to-face human feedback, which was providedthrough an individual writing conference, was investigated. The findings of this study can offer pedagogicalsuggestions for incorporating AES in L2 writing class and implications for future research in EFL context.

참고문헌 (Reference)

1 Ferris, D., "The “grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime...?)" 13 : 49-62, 2004

2 Chen, C. F., "The use of a computer-based writing program: Facilitationor frustration?" 2006

3 Montague, N., "The process oriented approach to teaching writing to second language learners" 10 : 13-24, 1995

4 Dikli, S., "The nature of automated essay scoring feedback" 28 (28): 99-134, 2010

5 Shermis, M. D., "The impact of automated essay scoring on writing outcomes" 2008

6 Chandler, J., "The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing" 12 : 267-296, 2003

7 Liu, Y., "The effects of error feedback in second language writing" 15 : 65-79, 2008

8 Raimes, A., "Techniques in teaching writing" Oxford University Press 1983

9 Barkaoui, K., "Teaching writing to second language learners: Insights from theory and research" 40 (40): 35-48, 2007

10 Othman, S. B., "Student response to teacher feedback on multiple-draft compositions in ESL classroom" 2009

11 Pasand, P. G., "Process-product approach to writing : The effect of model essays on EFL learners’ writing accuracy" 2 (2): 75-79, 2013

12 Krashen, S., "Principles and practice in second language acquisition" Pergamon Press 1982

13 Chung, K. W. K., "Methodological approaches to online scoring of essays" University of California, Center for the Study of Evaluation. (ERIC Document Reproduction Service No. ED 418 101) 1997

14 Lee, I., "L2 writing teachers’ perspectives, practices and problems regarding error feedback" 8 : 216-237, 2003

15 Hyland, K., "Feedback on second language students’ writing" 39 (39): 83-101, 2006

16 Hyland, K., "Feedback in second language writing: Contexts and issues" Cambridge University Press 1-20, 2006

17 Vokic, G., "Feedback in L2 writing : The students’ perspective" 9 : 139-155, 2008

18 Attali, Y., "Exploring the feedback and revision features of Criterion" 2004

19 Ferris, D., "Error feedback in L2 writing classes : How explicit does it need to be?" 10 : 161-184, 2001

20 Hyland, F., "ESL writers and feedback : Giving more autonomy to students" 4 (4): 33-54, 2000

21 Burstein, J., "CriterionSM online essay evaluation: An application for automated evaluation of student essays" 2003

22 Beuningen, C. V., "Corrective feedback in L2 writing : Theoretical perspectives, empirical insights, and future directions" 10 (10): 1-27, 2010

23 Chen, C. F., "Beyond the design of automated writing evaluation:Pedagogical practices and perceived learning effectiveness in EFL writing classes" 12 (12): 94-112, 2008

24 Huang, S. J., "Automated versus human scoring: A case study in an EFL context" 11 (11): 149-164, 2014

25 Toranj, S., "Automated versus human essay scoring : A comparative study" 2 (2): 719-725, 2012

26 Toranj, S., "Automated versus human essay scoring" 33 : 31-35, 2012

27 Jiang, J., "Automated scoring research over 40 years : Looking back and ahead" 5 (5): 56-63, 2012

28 Burstein, J., "Automated essay scoring for nonnative English speakers (PDF)" 1999

29 Dikli, S., "Automated Essay Scoring" 7 (7): 2006

30 Valenti, S., "An overview of current research on automated essay grading" 2 : 319-330, 2003

31 Dikli, S., "An overview of automated scoring of essays" 5 (5): 2006

32 Zhang, J., "A comprehensive review of studies on second language writing" 12 : 89-123, 2008