I came across this article recently, about artificially intelligent marking machines. Whilst not as terrifying as the corpse-eating battlebots which have also been in the news, my initial reaction was still one of distaste. But the more I thought about it, the more ambivalent I became.
People like tests. Students like to have a piece of paper which shows how good they are. Employers and schools like to see pieces of paper which demonstrate clearly that the candidate is up to the linguistic reigours demanded of them. But the whole system is an illusion. We have the hugely popular Test of English for International Communication which requires not a peep from the examinee – a multiple choice listening, vocab and grammar test using only native speaker varieties of English. Where is the “international” or the “communication” in that?
You might favour something from the Cambridge suite then? At least the candidate has to demonstrate a degree of oral proficiency. But if you’ve ever been through the examiners standardisation process, you’ll realise how deadly difficult it is to pick a 3.5 from a 4 in the PET Oral Exam. Those tests are rubric-ed to within an inch of their lives, effectively making the examiners robots.
As assessors of language, we are torn between two urges. We might start marking our stack of writing papers with absolute adherence to our carefully composed marking scheme, but two-thirds of the way down the pile it has all become much more impressionistic. Should we be unfailingly consistant by sucking the life out of our assessment, or mark holistically and let a few students get better or worse grades than they perhaps deserve.
Let’s face it, testing is a bit of a crapshoot. Are the placement tests we have now always accurate? I would doubt that anyone has had a class in which every student had been appropriately placed. So why not let the robots have a crack so we can get on with the teaching?