Evaluation
| OER Handbook for Educators | |
|---|---|
| Use | Integrating OERs in teaching and learning | Learner perspective | Educator perspective | Using Mobile Versions | Evaluation | Accessibility | Perspectives |
Previous chapters have discussed the concept of quality in OER production. Once you have used the OER, you should take the time to evaluate its quality. This recommendation may seem obvious, but evaluation is often neglected as time constraints often push educators to the next task. Simple OER, such as using a single page from a MIT course, may only need a basic evaluation. Thorough evaluation for complex OER can take enormous amounts of time and be difficult to set up. However, it is critical that you determine whether or not the OER is effective in the classroom.
Methods of evaluation
There is no single method of evaluating OER quality or its effectiveness in the learning activities involving it. For many educators, the most important thing to measure is the learning outcomes. This part of evaluation is routine, since you are already evaluating learners on what they have learned. Although learners failing to acquire the knowledge and information does not mean the OER is faulty, it does raise questions about its effectiveness.
Another metric for evaluation is learner reaction. In addition to finding out whether or not learners liked the OER, find out the "whys" behind their preferences. Although the composition of classrooms change over time, you should start to see patterns in the preferences of students. This evaluation can take the form of a paper survey, in-class discussion or focus groups. Which method you chose will depend on the time you are able to devote to evaluation.
The third metric is a difficult one to measure, but it is what is often called "return on investment (ROI)." The concept of return on investment essentially asks "Was it worth the investment?" In order for measurement to be fully accurate, you need take into consideration the time taken at each part of the OER life cycle. This metric is largely subjective, as only you can measure how much your time is worth. You'll probably find that your first OER will take more time than you originally thought. It is not uncommon to have technological issues during the first implementation. This should not discourage you from future OER production and use; as you develop new skills and refine others the amount of time needed will be reduced. You should also consider how much time it would have taken you to build the OER from scratch in relation to the other costs of proprietary solutions.
Consider submitting your OER to a repository that offers ratings and comments. Though these ratings and comments take time to accumulate, it can be a good way to poll the opinions of others. Popular repositories for rating might include:
- Wikipedia[1]: add to an existing article or create a new one.
- WikiEducator[2] and Wikiversity[3]: add a new project and connect with fellow educators.
- Flickr[4]: no ratings, but users frequently comment on photos and some users have groups that recognize outstanding photos.
- Kaltura[5] / TeacherTube[6] / YouTube[7]: registered users can rate and comment. YouTube's recommendation system will suggest similar videos. Kaltura allows others to edit and make improvements to the video.
- ccMixter[8] and The Freesound Project[9]: upload music to ccMixter and sounds to Freesound project. Commenting systems on both sites.
You should not be afraid to decide against developing a particular OER project in the future. There are many good ideas for OER projects that simply are not feasible at this point in time given the amount of time needed. Start with simple OER exercises and projects and take on more complex ones as your confidence grows over time.
