An exploratory study into automated précis grading

Publication type
P1
Publication status
Published
Authors
De Clercq, O., & Van Hoecke, S.
Series
PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020)
Pagination
397-404
Publisher
European Language Resources Association (ELRA)
Conference
12th International Conference on Language Resources and Evaluation (LREC) (Marseille, France)
Download
(.pdf)
View in Biblio
(externe link)

Abstract

Automated writing evaluation is a popular research field, but the main focus has been on evaluating argumentative essays. In this paper, we consider a different genre, namely précis texts. A précis is a written text that provides a coherent summary of main points of a spoken or written text. We present a corpus of English précis texts which all received a grade assigned by a highly-experienced English language teacher and were subsequently annotated following an exhaustive error typology. With this corpus we trained a machine learning model which relies on a number of linguistic, automatic summarization and AWE features. Our results reveal that this model is able to predict the grade of précis texts with only a moderate error margin.