An exploratory study into automated précis grading

Publication type
C1
Publication status
Published
Authors
De Clercq, O., & Van Hoecke, S.
Series
12th International Conference on Language Resources and Evaluation Conference (LREC 2020), Proceedings
Pagination
397-404
Publisher
European Language Resources Association (ELRA) (Marseille, France)
Conference
The International Conference on Language Resources and Evaluation 2020, LREC 2020 (Marseille, France)
Download
(.pdf)
View in Biblio
(externe link)

Abstract

Automated writing evaluation is a popular research field, but the main focus has been on evaluating argumentative essays. In this paper, we consider a different genre, namely précis texts. A précis is a written text that provides a coherent summary of main points of a spoken or written text. We present a corpus of English précis texts which all received a grade assigned by a highly-experienced English language teacher and were subsequently annotated following an exhaustive error typology. With this corpus we trained a machine learning model which relies on a number of linguistic, automatic summarization and AWE features. Our results reveal that this model is able to predict the grade of précis texts with only a moderate error margin.