By Barbara Metcalf, Teaching Faculty Member, Bachelor of Nursing Program, Faculty of Health and Human Sciences, VIU

For all blog posts on this topic see Calibrated Peer Review Part I, Part II, Part IV and Part V

 

 

I thought I would start this posting by describing the Calibrated Peer Review (CPR) process after which our process was modeled. While this is used in several institutions of higher learning, our process was modeled after the CPR process used at Alverno College in the States. The Alverno College uses an automated system where the drafts are submitted online. Before a student can provide a mark or feedback to a peer, they must mark a couple of papers online. If the mark they give is not close to what the instructor would have given the paper, they must mark another and another until they get it right. That’s the ‘calibrated’ part of the peer review. Once they are marking close to what the teacher would have done, they get the actual paper they are to mark. Another peer also gets the same paper to mark. This is all done electronically. The instructor is only notified when the two peer marks for the same paper are widely disparate. I believe that in the Alverno model, the final papers are marked the same way. The instructor does not do any of the marking. I know that many instructors would have difficulty letting go of the marking of the final paper, but after my experience with the marking involved in our rendition of CPR, I would be quite happy letting that go.

For our experiment in CPR, I found the marking to be quite onerous. I had to mark all the drafts (66 in all), all the feedback on the drafts (132 in all), all the final papers (66) and all the reflections on the CPR process (66). The papers were such poor quality that I spent roughly an hour per paper. And that was after spending about ½ hour per draft to give them a preliminary mark for comparison. At 66 papers, that was a great deal of time. I felt that, since the entire point of this was to give good, quality feedback to help them move forward in their writing, that I should be spending that time. There was still a final project that required marking after this that came due in Week 11 of the semester, so I was marking solid from Week 6 until week 14 with one week off in the middle for good behaviour. Everyone, students and instructor, were bending under the workload. So should I be involved in this again next year, I will definitely rethink this and come up with something to decrease this workload. I know that Liesel is looking into acquiring the software to make it closer to Alverno’s process and that would lighten the load considerably. I would say that the majority of the issues I had with CPR revolved around the workload, and these would all be fixed with an electronic process.

 

The compressed time line lead to some interesting findings for me. In years past, the final paper was due at the end of the term, with a different project due earlier. For CPR, I had to switch these around. This year, with having the paper so early to accommodate the CPR timetable, I found the papers to be of a relatively low quality overall. I think many embraced the fact that it was a draft and perhaps meant to work on them before handing in the final, but many things were due at that time and the majority showed very little change from the draft. I don’t know if this is the kind of process that usually sees rewards immediately, but it was not so with these groups. Perhaps future classes will reap the rewards of better writing. What was interesting though is that the project that was now due at the end of the term was really well done overall. This had been reversed last year with the marks and quality of the project being lower than the final paper’s. I’m not sure what to make of that, but I am mulling it over.

 

One of the difficulties I found with having the paper early in the term, is that students had to write a paper on therapeutic communication techniques before they even knew what therapeutic communication techniques are. Even after telling them in class what was and was not a communication technique, I was still getting ‘rapport’, ‘boundaries’ and ‘phenomenology’ as communication techniques. If we do this again next year with the same time line, I will have to be more prescriptive and give them a list of techniques from which to choose.

In future submissions, I will share some of the thoughts the students had one the CPR process and my thoughts one how this entire process could be tweaked for next year.