Suppose you have given your students an interesting task in which they have to integrate and adapt knowledge from your lessons (for example, writing an opinion piece about organ donation). The deadline approaches, the tasks trickle in and you start assessing. Task 1: strong content, well written and not too many language mistakes. Your decision? A 9! Task 2: fluent writing style, just a few language errors, but what a sound argumentation! Decision? Doubts set in. Are those language errors the deciding factor and do you give an 8? But this argumentation was really stronger than in task 1 and it already received a 9… Recognisable? This blog explains why assessing is so difficult.

And let’s keep things clear from the start. Assessing is especially  difficult when it comes to such open and generic tasks. Closed questions, fill-in-the-blank questions or multiple choice questions do not usually cause the same doubts. Such questions are aimed at the reproduction of knowledge rather than the demonstration of competences. If you are going to assess open tasks anyway, then you often run into the following difficulties:

1. You know the variation in scores, but not the variation in quality

A judgement is always a personal impression of quality. This is because there is a wide range of possible scores and also a wide range of quality. And while you can think of all the possible scores, you have no idea of all the possible variations in quality. So you are completely in the dark about which score goes with which quality. And that, of course, is difficult.

2. You assess differently to your colleagues

Because assessment is a personal impression, this impression differs for different people with different personalities. Perhaps you are much stricter than your colleague, or maybe you value other aspect in those open tasks. You know these differences exist, and that makes you insecure.

3. Your judgement depends on the moment

Because let’s face it, after a nice snack you’re probably a different person than when you’re very hungry, right? Or in the morning after a good night’s rest compared to an evening after a busy day at work? Judgements also differ between moments. Somehow we know that. And that is not a nice feeling.

4. And the other tasks matter too!

And as if all of this is not making us insecure enough, the order of the tasks also matters. If you read a very good task first, the tasks that follow might not live up to your (higher) expectations. And vice versa, If the first task really needed improvement, you might be more lenient for the tasks that follow. So you look back and forward, adjust you judgements, and above all, start doubting – again.

By now, we all agree that assessing is difficult. Fortunately, research shows us what causes this. And it turns out that we very often don’t judge in a way that fits in with how our brain works. Compare it to buying a pair of shoes. Have you ever walked into a shoe shop with a list to score different aspects of shoes? Just what I thought. Rather, what we do is compare pairs of shoes or compare them to an ideal image in our head. It’s the same with assessing those open tasks. It is much easier if you compare them with each other. This is precisely why comparative judgement is such a nice method. It makes your assessments simpler, and therefore more reliable and valid.

Want to know more about comparative judgement? send us an e-mail or request a workshop!