OERu/Strategic plan 2014 - 2017/Strategic plan 2015 - 2017/Measuring progress
How we measure progress
At the OERu 2011.11 meeting of founding anchor partners it was proposed that the project evaluation be guided by the Context, Input, Process and Product Evaluation (CIPP) model (see Stufflebeam 2007) with the aim of facilitating decision making and providing input into the structure of the OERu project framework as it evolved. The following diagram developed by Stone and Lane (2012) provides a comprehensive yet succinct overview of how the CIPP Model can inform the four major decision phases of the project cycle:
The essential focus of each of the four phases of the CIPP model was described by Stufflebeam (1971) as follows : “Context evaluation serves planning decisions by identifying unmet needs, unused opportunities and underlying problems that prevent the meeting of needs or the use of opportunities ; Input evaluation serves structuring decisions by projecting and analysing alternative procedural designs; Process evaluation serves implementing decisions by monitoring project operations; Product evaluation serves recycling decisions by determining the degree to which objectives have been achieved and by determining the cause of the obtained results” (p. 268). Source: Stufflebeam, D. L. (1971). The use of experimental design in educational evaluation. Journal of Educational Measurement, 8(4), 267-274.
The Context Evaluation of the OERu was successfully completed in 2012. A summary of the results is available on Slideshare.
While the current focus is on the input evaluation phase, it is important to acknowledge that the context of the OERu project is continually evolving. Based on his analysis of different decision making settings, Stufflebeam (1971) would probably have classified the OERu context as “neomobilistic” demanding decisions that involve large innovative efforts aimed at solving significant problems. The OERu project is complicated further by the gradual addition of new partner institutions, each of which is at a different stage of designing and developing their particular contributions.
Fortunately, the input evaluation focusses on decisions associated with the resources, tools and methods available for achieving OERu goals across institutional settings and is therefore able to accommodate institutional diversity. Coordinated by the Australian Digital Futures Institute at the University of Southern Queensland, the team of Dr. Xiang Ren and Professor Mike Keppel, has delineated a list of draft questions for the input evaluation to enable feedback and refinement prior to, and during, the forthcoming 3rd meeting of OERu partners in Hobart from 6-7 November, 2014. The outcomes of such stakeholder consultation will lead to the implementation of a survey of OERu partner institutions early in February 2015, followed by in depth interviews with selected representatives from each of the OERu Working Groups. It is anticipated that the Input Evaluation Report will be available for circulation and comment by 30 April 2015. The input evaluation findings will assist partner institutions to devise a pragmatic program strategy that is economically, socially, politically, and technologically feasible. This work will also lay the foundation for subsequent process and product evaluations.
