17 Feb Product development students evaluate videos with Comproved
Ivo Dewit is lecturer in the course Product-service system design in the Product Development master at the University of Antwerp. During the covid-19 period, he started working with Comproved out of necessity. He has been hooked on the tool ever since. His students evaluate each other’s work using videos in the tool. We were curious and asked him about his experiences.
How did you learn about Comproved?
“During the covid-19 period, I was looking for a cool, interactive way to work with my students, whom I could no longer see live at the time. I wanted to recreate the feeling of being in the design studio, but in an online environment. When I was exploring the different tools on Blackboard, I bumped into Comproved.”
“Coincidentally, we had someone at the faculty in charge of quality and innovation who had contacts at Comproved. He then urged me to give the tool a try. I was an instant fan.”
Previously, we did peer evaluations within groups, but with Comproved, students also evaluate between groups. This allows them to see each other's work, see good and not so good products and accordingly assess their own work's quality.
What do you use the tool for?
“I use it as a peer assessment tool. What I like about it is that students get an idea of what others are doing. Before, we did peer evaluations within groups, but with Comproved, students also evaluate between groups. This way, they can see each other’s work, see good and not so good products and accordingly assess their own work’s quality. Furthermore, I let the students not only give each other feedback, but also feed up and feed forward, which they can really use to improve their work.”
Was it easy to convince your colleagues to work with the tool?
“I am the coordinator of the course so I really only had to impose it on my colleagues (laughs). At the same time, it helped that the tool is evidence-based and that it produces nice results. For teachers, learning to work with the tool requires an effort in the beginning, but you receive a lot in return.”
“What was an advantage during the covid-19 period was that by using the tool, the teachers were much better informed about the students’ work. That made it easier to discuss the results with them afterwards.”
Can you tell us something more about the assignment you evaluate with Comproved?
“In the first master year of Product Development, students have to design a product-service system in groups of 4-5. A product-service system consists of a product, an interface in the form of an app or an online platform and a service. Think, for example, of Velo Antwerp’s shared bicycles. So in 12 weeks, they have to design something like this. Each week they go through a particular design process with specific tools and methods. This way they work step by step on a number of deliverables.”
“During the covid, the students had to assess each other’s deliverables in Comproved every week. That way, they learned a lot from each other. Now they only have a midterm assessment in Comproved halfway through the semester. Each group then has to upload a 5-minute video introducing their project. They then evaluate each other’s videos and give peer feedback that they can use afterwards to improve their work.”
How do you introduce the tool to students?
“At the beginning of the semester, I give an introduction to the course. Among other things, I explain the theme of the year and we form the groups. I also explain how the course will be evaluated. Using some screenshots, I explain what Comproved is, how to make comparisons and give feedback. Finally, they get to see some examples from previous years.”
“When it is then time to start working with Comproved, I also guide the students a bit so that as little goes wrong as possible. For example, I remind them to embed a YouTube link to their video and not upload an mp4 because the latter loads too slowly. Or I monitor that everyone submits on time and help where necessary.”
How does the feedback process go?
“Giving good feedback remains difficult. We find that students often give feedback on the more superficial elements such as the sound quality of a video or the use of language. So we provide guidelines to challenge them to formulate some deeper feedback. For example, we ask them to look at the video from the end customer’s point of view. What would they think of the design and how it is communicated? What suggestions or comments would they have to improve the design?”
“We also remind students that it is really okay to be critical and strict with each other in the peer feedback. The students know that that midterm evaluation does not count towards their final subject score. So the groups that perform less well get a chance to recover. In most cases, they also do.”
Students really like receiving more feedback than when they only get teacher feedback. They also find it insightful to know their place in the rankings. This can be confrontational, but they learn a lot from it.
How do the students experience working with the tool?
“Students really like receiving more feedback than when they only get teacher feedback. They also find it insightful to know their place in the rankings. This can be confrontational, but they learn a lot from it. I notice that they are activated by looking at the higher-ranked products and the feedback. They then step up a notch and so the bar is raised a little higher again.”
“I once compared the grades we give with the colleagues and the grades coming from a peer assessment in Comproved. Those are very hard in line with each other, so that is statistically quite correct. That’s nice to see and the students also see that that’s correct.”
What do you think are the biggest advantages of the tool?
“I think the biggest advantage is that students learn a lot from peer assessment. Along the teacher side, I like that you get a lot of extra information behind the scenes. How much time do students spend evaluating? How much feedback did they give? How many comparisons did they make?”
“I also find the ranking useful to consult prior to a feedback discussion with a student group. We used to divide the groups among teachers and you were only up to date with the groups you were evaluating. Now we can see all the videos in Comproved, the ranking and the feedback received. So you are much better informed.”
“What I also like about the tool is that you can evaluate different file types. We evaluate videos, but you could have students upload concept sketches, for example. That’s super easy. Which do you think is the best sketch? Why? That way, students also learn the fun of evaluating. That works better than discussing work in class or as a group.”
Do you have some tips for teacher who want to start working with Comproved?
“I would say: just do it! With trial and error, you learn to work with the tool. It’s also not a disaster if something goes wrong once in a while. Your students will forgive you for that. Moreover, they will soon realise that the return of an assessment in Comproved is worthwhile.”
“What can be useful is to first organise a test round with colleagues to experience how everything works. Or you can have your students do a test assessment in class in which they assess and provide feedback on 3 examples from a previous year. That way, you quickly know where problems arise and how to solve them.”
Are you inspired by Ivo’s story and also want to get started with Comproved? Read on here or contact us and we’ll help you get started!
