From WikiEducator

A lot of journals and readings used in the course are openly available. However if they are not, journals and books for this subject are available from the Manukau Institute of Technology library.

Contents 1 APA Referencing 2 Must have material 2.1 Online books 2.1.1 Benchmarking eLearning material 3 Just in - General material about evaluation 4 Presentations 5 Literature 5.1 Journals 5.2 Journals - Material just in 5.3 Conference proceedings 5.4 Articles 5.5 Reports 5.6 Newsletters 6 Evaluation Tools

APA Referencing

APA (American Psychological Association) referencing is used in this course. There are lots of links to resources about this style of referencing on the Manukau Institute of Technology library website. The MIT Guide is very useful.

Must have material

This is the material we will be referring to a lot. It is listed here so you can find it easily. The other material further on may be useful and may have been referred to in previous sections.

• Course notes - A summary of some methods of evaluation.
  

Online books

• Hegarty, B. (2007) Evaluating the impacts of eLearning Chapter Eight in the eLearning Guidebook.
• Phillips, R., Bain, J., McNaught, C., Rice, M. and Tripp, D. (2000). Handbook for Learning-centred Evaluation of Computer-facilitated Learning Projects in Higher Education.
• Text book: Reeves, T. C. & Hedberg, J. G. (2003). Interactive Learning Systems Evaluation. Englewood Cliffs: Educational Technology Publications. Google books has a preview of the book including the Table of Contents.
• The companion web site makes available a full set of evaluation tools (forms and protocol).
• Online resource: eLearning Guidelines for New Zealand: http://elg.massey.ac.nz/
• Online book: Evaluation Cookbook to decide on some sampling methods for your project.
• Online resource: Project Evaluation Toolkit from the University of Tasmania.
• Online resource: Evaluation Framework.

Benchmarking eLearning material

• Maturity Model Getting Started - this is relevant to the NZ tertiary sector.
• Marshall, S. (2006). E–learning Maturity Model Process Assessment Workbook. There are five main areas for assessing quality and benchmarking eLearning:

• 1. Learning Processes that directly impact on pedagogical aspects of e-learning;
• 2. Development Processes surrounding the creation and maintenance of e-learning resources;
• 3. Support Processes surrounding the oversight and management of e-learning;
• 4. Evaluation Processes surrounding the evaluation and quality control of e-learning through its entire lifecycle;
• 5. Organisation Processes associated with institutional planning and management.

• Marshall, S. J. (2008). What are the key factors that lead to effective adoption and support of e-learning by institutions? In Proceedings of HERDSA 2008 (Rotorua, New Zealand, HERDSA).
• Chickering, A. & Ehrmann, S C. (1996). Implementing the Seven Principles: Technology as Lever. AAHE Bulletin, October, p. 3-6.
• IHEP (2000). Quality on the Line: Benchmarks for Success in Internet-Based Distance Education. The Institute For Higher Education Policy.
• Choy, Sarojni C. (2006). Benefits of e-learning benchmarks: Australian case studies.
• Institute for Higher Education Policy (1998). Assuring Quality in Distance Learning: A Preliminary Review.

Just in - General material about evaluation

This list will update as participants find new material and tag it in del.icio.us

• Please use the tag word "eval2010" to categorise and share what you find.

Here you can only see the latest five. To see the full list: http://del.icio.us/bronwynh/eval2010

The Deming Cycle (PDSA or PDCA)

One of the most common tools for improvement is the Deming (or Shewhart) Cycle. This method is also known as Plan-Do-Check-Act (PDCA) or Plan-Do-Study-Act (PDSA) and it is well suited for many improvement projects. Originally known as PDCA, it was revised to PDSA in the early 1990’s to reflect misunderstandings of the intent of learning process. [?]

Strategic Evaluation

A useful way of looking at evaluation is to distinguish between approaches, purposes, methods and designs. model by Duigan 2001 [?]

Instructional Design Models & Theories | Instructional Design Central

The four levels of Kirkpatrick's evaluation model essentially measure: * REACTIONS-What they thought and felt about the training * LEARNING-The resulting increase in knowledge or capability * TRANSFER-Extent of behaviour and capability improvement and implementation/application * RESULTS-The effects on the business or environment resulting from the trainee's performance [?]

Instructional Design Using the Addie Model

ADDIE Model The acronym "ADDIE" stands for Analyze, Design, Develop, Implement, and Evaluate. It is an Instructional Design model that has withstood the test of time and use. It is simply a "device" to help us think through a course's design. Though the model appears linear, it does not have to be followed rigidly or in a linear approach, especially if you already have course materials developed. The table below gives an abbreviated overview of some of the components of ADDIE. [?]

Distance Education Quality Principles

includes - Quality on the Line: Benchmarks for Success in Internet-Based Distance Education A report released in March 2000 by the Institute for Higher Education Policy (IHEP) Chickering seven principles and much more. [?]

Presentations

• Bonnie and Suzy DVD in Nursing evaluation presentation on Elluminate. The full title of the presentation is: Evaluating the effectiveness of a DVD in the flexible mode of delivery in nursing education.
• Presentation only with synced audio Evaluating the effectiveness of a DVD in the flexible mode of delivery in nursing education with Bonnie and Suzy.
• The impact of a Learning Management System and the ways in which academic staff transferred their learning from one system to another was investigated by Selena, Trish and Lois. View the presentation made by Selena which is synched with audio.
• A presentation of summative evaluation conducted by Sandra Elias looking at online discussion which was used for assessment in a nutrition course in a midwifery degree.

Literature

Journals

• Australian Educational Computing. This is the refereed journal of the ACCE (Australian Council for Computers in Education) and is published twice a year.
• Evaluation issue of the Australian Journal of Educational Technology (AJET) has lots of evaluation articles. (2002). Vol 18(2).
• AJET homepage with access to all the online journal articles.
• Education-line is an archive of bibliographies and full-text, online reports. It is a joint project of the British Education Index and a consortium of British professional educational associations.
• The European Journal of Open and Distance Learning (EURODL). "An online journal on distance and e-learning, publishes the accounts of research, development and teaching for Europe in its most inclusive definition, exploring the potential of electronic publishing. Furthermore, EURODL presents scholarly work and solid information about open, distance and e-learning, education through telematics, multimedia, on-line learning and co-operation."
• First Monday online journal.

Journals - Material just in

This list will update as participants find new material and tag it in del.icio.us

• Please use the tag word "journaleval2010" to categorise and share what you find.

Here you can only see the latest five. To see the full list: http://del.icio.us/bronwynh/journaleval2010

International Journal of Emerging Technologies in Learning (iJET)

open journal with relevant material, for example, Vol 4, No 1 (2009) article on quality assurance - Breaking Boundaries: Quality E-Learning for the Global Knowledge Society by Lalilta Rajasingham pp. 58-65 [?]

Online International Journal of Usability Studies - Do Usability Expert Evaluation and Testing Provide Novel and Useful Data For Game Development?

A case study was done to study whether usability expert evaluation and testing are suitable for game development. In the study, a computer game under development was first evaluated and then tested. Game developers were then asked to rate the findings and [?]

Conference proceedings

• Australasian Society for Computers in Learning in Tertiary Education ASCILITE]has an annual conference and online proceedings.

"It is a society for those involved in tertiary computer-based education and training, including educational interactive multimedia. It provides a forum to stimulate discussion of relevant issues in the educational use of technology, as well as promoting research and evaluation."

• Evaluating the New Teaching Technologies Workshop (2000). proceedings.
• Online Learning and Teaching Conference (2004) proceedings.
• UniServe Science Symposiums proceedings.

Articles

• Bates, Paul & Obexer, Regina. (2005). Evaluating student-centred teaching and learning strategies for Aviation students using a quality framework for online learning

environments. In ASCILITE 2005 conference proceedings: Balance, Fidelity, Mobility: Maintaining the Momentum.

• Brittain, Sarah, Glowacki, Pietrek, Van Ittersum, Jared and Johnson, Lynn (2006). Podcasting Lectures: Formative evaluation strategies helped identify a solution to a learning dilemma. This is an article of an evaluation study conducted at the University of Michigan (U-M) School of Dentistry.
• Elgort, I., Smith, A. G. & Toland, J. (2008). Is wiki an effective platform for group course work? Australasian Journal of Educational Technology, 24(2), 195-210.
• Muirhead, R.J. (2007). E-learning: Is this teaching at students or teaching with students? Nursing Forum 42(4),178-184.
• Phillips, R. (2005). We can’t evaluate e-learning if we don’t know what we mean by evaluating e-learning! Interact, 30, p 3 - 6. Learning Technology Support Service, University of Bristol.
• Rourke, L., Anderson, T., Garrison, R. & Archer, W. (2001). Assessing Social Presence In Asynchronous Text-based Computer Conferencing. Journal of Distance Education/Revue de l'enseignement à distance.
• Sleezer, C. M., Kelsey, K. D., & Wood, T. (2005). Assessing safety training needs at AVN 300: A case study. Proceedings of the Canadian Evaluation Society and the American Evaluation Association Joint Conference, October 24-30, 2005. Toronto, Ontario, Canada. An article about a needs analysis - demonstrating Stake's Responsive Evaluation Model.

Reports

• Online information literacy project reports (scroll down to documents) at: Online information literacy project. You will note some reports are brief, e.g., Prototype Evaluation Report, and others such as the final report for the project are more detailed.
• Winter, M. (2010). Second Life Education in New Zealand: Evaluation Research Final Report .
  

Newsletters

• Implementing e-learning guidelines newsletter for April 2008.

Evaluation Tools

• Explore the

• Take a look at some
• An example of how results could be laid out and the type of analyses which can be done – median, frequencies in this summative evaluation report of results.
• Nielsen, J. (1994). Ten Usability Heuristics.