Comments from Brian
Response to Brian's feedback
Brain - thanks for that feedback - valuable ideas and foundations we need to incorporate into the design of the fist step in the project.
An important facet of this opportunity for comment is to gain sufficient information to help Akash develop specifications for the first prototype. We follow an incremental design approach at the OER Foundation whereby we focus on small steps of implementable code following a learn by doing approach.
The idea of developing mathematical models to determine the "confidence of the reviewer" over time is important, similar to Jim's concept of developing a karma system.
Once we get to the point of implementing a specification for the first prototype, I would like to make a call to OERu partners to identify one or two statisticians or applied mathematicians who could advise on decisions for the mathematical model for the first iteration.
We will also need to be realistic - the GSoC project only allows for 3 months of coding and its unlikely that we are going to be able to build a solution which is going to address all the complexities associated with peer evaluation systems. However, I guess that we will be able to build something useful as a first step towards the next iteration.
