Peer evaluation
I agree with Jim, the peer evaluation sounds interesting and is a necessary component for scaling OERu courses in the future.
One of the unique aspects of an open online course, is that learners can choose how they participate. From low level "sip and dip" "read only" to active engagement participating in all the activities. To do date, the evaluation (validation) of the blog posts are used for certification of participation rather than course grades. (That's not to say that peer evaluation for some elements of grading will be excluded in the future.)
You will need to have a think about an implementation model for assigning learners to review designated blog posts (given that not all learners will be posting blogs and the problem of a few posts getting most of the reviews). One implementation is to only assign the peer review task to those learners who have already posted blogs themselves, for example allocating the two most recent posts to a learner for review and somehow notifying the learner of the reviews they should complete on their course dashboard. We will need to think about what happens when someone is assigned a blog to review, and doesn't complete the task. We could for example design a reward system where a blog reviewer gains points which are recognised as a course participation metric for competing their assigned reviews. We will need a mechanism to know which posts have been reviewed and consider the option for minimum number of reviews per post.
I have given a bit of a thought on the peer evaluation workflow.
- The peer evaluation dashboard will be very similar to the Activity register for a course, like http://wikieducator.org/Open_content_licensing_for_educators/E-Activity_register but with the following changes:
- Only posts for an activity for which he has submitted himself can be seen to him.
- Only posts which have less than a fixed number of reviews will be visible in this dashboard, for example a course can be set as each activity requiring about 2-5 peer reviews. So, only the posts with less than 5 reviews will be displayed in this dashboard.
- Posts will be sorted according to the number of peer reviews, and then newest first, i.e, the lastest posts with no peer evaluations yet will appear higher in the list.
- The table will look like https://www.dropbox.com/s/vkmvxj4kx73ekjd/IMG_20140308_221215.jpg (a very rough diagram.)
- The user can then click on a particular row to go to the peer evaluation page of that post.
- It will contain a link to the actual blog, where the learner can go and have a look at it.
- It will look something like https://www.dropbox.com/s/itggpqb7z8hmzmp/IMG_20140308_221558.jpg (a very rough diagram.)
- Now, the course can set a minimum number of posts that each user has to review for each activity,say 3 or 4,this will not only serve the purpose of evaluation but will increase learning as learners will be reading the posts of their peers, without taking away to freedom of which posts they are to review. They would get some credit for each post they review.
The above workflow assures that all posts would be peer reviewed as well as the learner will have freedom to choose the posts they would like to read from the ones which have not yet been evaluated.
Because we don't have a clear way to tie together a user and a natural person, self-assignment allows the creation of sock-puppets for grading or favorably marking friends. At the same time it allows smaller self-organized cohorts to work within a larger class. (My own experience in MOOCs suggests the latter is helpful and should be encouraged.)
I actually think all the submissions should be visible, with those needing review being highlighted. I don't want to prevent a motivated student from reading everything (not that he is likely to review everything).
It would be important to record the time of the review (since remote content could change after review).
I think some sort of peer review is important to the Academic Volunteers International scheme. It also suggests there might be multiple types of reviewers, possibly categories like: current student, student that has completed the course, tutor, community volunteer.
