These are the main advantages of comparative judgement

What teachers sense from their experience is confirmed in many scientific studies: when it comes to complex skills, traditional assessment methods are often insufficient. For instance, as a teacher you can work your way through an entire criteria list or rubric, but still feel that the final score does not match how you assess the work’s overall quality. Recognizable? Fortunately, there is an alternative. Research shows that comparative judgement is a reliable method for assessing competences. But what exactly are the main advantages of comparative judgement? We will tell you in this article.

1) Reliable results

Research has shown that comparative judgement leads to very reliable results. This is because comparing is cognitively an easier task than, for example, assessing using a criteria list. As a teacher, when you compare tasks in pairs, you will – thanks to your expertise – be able to answer effortlessly which piece of work is the better of the two.

Because comparing is easier, you will also make more consistent decisions. Therefore, the same piece of work will stand out in a comparison every time, regardless of the time of day or which tasks you have seen before. So, as an assessor you can be pretty confident about your judgement.

2) High validity

Validity has to do with the fact that you are effectively assessing the competence that needs to be assessed. However, the problem with complex competences is that they usually cannot be captured in too tight a framework. Research shows that analytical assessment methods have a lot of overlap between the aspects to be distinguished. Moreover, you push the bigger picture out of sight by zooming in on the separate parts. All this makes assessing difficult and inefficient.

By assessing in a more holistic way, which is the case with comparative judgement, assessors automatically take more criteria into account. Even those criteria that are not made explicit in criteria lists or rubrics, but that are nonetheless relevant. Moreover, by assessing together with some colleagues and combining different perspectives, the different (minor) aspects of a competence are all being taken into account. Thus, you assess the competence in its entirety.

3) Efficient

Another of the advantages of comparative judgement is that it saves you a certain amount of time. Don’t get me wrong, grading is a time-consuming job, whichever way you approach it. However, if you think that comparative judgement costs you twice as much effort because you assess in pairs, you are not quite right either. For pairs with a clear difference in quality, it is quickly obvious which product is the best. If the products are of similar quality, then things will logically be slower. But the overall assessment time will usually be no longer than with analytical assessment.

The biggest time gain in comparative judgement? No criteria list needs to be developed, validated and calibrated. Comparative judgement relies on the expertise of assessors, which has proven to be very reliable. Moreover, it works intuitively. Assessors do not need to be trained to learn to look at (the same) aspects.

4) Extra learning opportunities for students

However you use the method, it provides great learning opportunities for students anyway. For instance, after all the products have been compared, you could have students look at the ranking of the works. This will help them understand where they themselves are situated in the rank order. Since they get a chance to look at better and lesser examples of the task, they discover why their work differs in competency level.

When you use the method for peer assessment, even more learning opportunities are added. For example, (anonymously) assessing peer work is a learning opportunity in itself. Indeed, by comparing the works, students learn bottom-up to recognize key aspects in qualitative work. By explicitly naming these in the feedback they give their fellow students, they activate this knowledge in themselves as well, which (hopefully) benefits their follow-up works.