I came across this article recently, about artificially intelligent marking machines. Whilst not as terrifying as the corpse-eating battlebots which have also been in the news, my initial reaction was still one of distaste. But the more I thought about it, the more ambivalent I became.

People like tests. Students like to have a piece of paper which shows how good they are. Employers and schools like to see pieces of paper which demonstrate clearly that the candidate is up to the linguistic reigours demanded of them. But the whole system is an illusion. We have the hugely popular Test of English for International Communication which requires not a peep from the examinee – a multiple choice listening, vocab and grammar test using only native speaker varieties of English. Where is the “international” or the “communication” in that?

 You might favour something from the Cambridge suite then? At least the candidate has to demonstrate a degree of oral proficiency. But if you’ve ever been through the examiners standardisation process, you’ll realise how deadly difficult it is to pick a 3.5 from a 4 in the PET Oral Exam. Those tests are rubric-ed to within an inch of their lives, effectively making the examiners robots.

As assessors of language, we are torn between two urges. We might start marking our stack of writing papers with absolute adherence to our carefully composed marking scheme, but two-thirds of the way down the pile it has all become much more impressionistic. Should we be unfailingly consistant by sucking the life out of our assessment, or mark holistically and let a few students get better or worse grades than they perhaps deserve.

Let’s face it, testing is a bit of a crapshoot. Are the placement tests we have now always accurate? I would doubt that anyone has had a class in which every student had been appropriately placed. So why not let the robots have a crack so we can get on with the teaching?

pixelstats trackingpixel

3 Thoughts on “do androids dream of electric test papers?

  1. Great article and a fine blog. I recently attended a conference at which the issue of testing spoken English electronically was raised. What was shown was both amazing and a little frightening, in terms of what can be achieved and also what can’t.

    I’ll be writing it up over on my blog soon.

  2. darren on October 28, 2009 at 4:55 pm said:

    Thanks Adam… I look forward to your post on the topic. I’m no expert in assessment, but I know that I would rather hand some of it over to robots!

  3. You are SO right! Most testing is inadequate and unrealistic and yet it seems to dictate the the whole syllabus. I mean, exam courses, what are they about? You do an IELTS course to show that your English is good enough to get into an English-speaking university and you spend ages learning to describe a graph because that is an exam class. But I can’t remember ever having to describe a graph at university. And there’s so much other language and so many other skills that areneeded at university, but if it’s not in the exam, students don’t want to do it. I say ban exam courses. Exam tasks should reflect learning tasks, not the other way round.

Leave a Reply

Subscribe without commenting

Post Navigation