WikiEdProfessional eLearning Guidebook/Evaluating the impacts of eLearning/Planning an evaluation

First Steps in Planning an Evaluation

Before an evaluation can be planned, the purpose for doing it must be clear, and the questions to be answered carefully thought out. This will help the evaluator decide on the type of evaluation to be conducted. Is the evaluation formative or summative? There are some interesting definitions on the table. Formative v.s. Summative Evaluation

For example, before a new multimedia product is introduced into a course, it is better to evaluate its suitability for target users before spending large sums of money on it. What might be ideal for one group of learners may be of little value to others.

Once the purpose and big picture questions are established, the evaluation methods and tools (instruments) can be selected. This will include the evaluation paradigm to be followed and the method you will use. As part of this, the type of sampling instruments e.g. questionnaires, checklist, interview questions etc. to be used to collect data need to be selected.

An evaluation plan might include the following headings:

• Introduction – informs reader about the project, who is involved and what will be included in the plan.
• Background and rationale – reasons the project is needed.
• Purpose – what the project intends to achieve and the ‘big picture’ questions.
• Methods – paradigm and methodology (participants, sampling instruments).
• Timeline/milestones - when aspects of the plan are to be completed.
• Example of sampling instruments - interview questions, survey - these are best located in the Appendix.
• References - list of literature, websites etc. accessed and referred to in the plan.

Activity

Prepare a plan for conducting an evaluation project in the workplace. (1500 words is about right.) Decide on the type of evaluation you will conduct and the product or programme to be evaluated. Hint: What is the purpose of the evaluation and the ‘big picture’ question(s) you want to ask? Decide on the evaluation paradigm and method you will use. Hint: Select the type of sampling instruments e.g. questionnaires, checklist, interview questions etc. to be used to collect data. If you would like feedback on parts of your plan, please post on the Discussion Forum for your course e.g. methods and sampling tools often need critique from another pair of eyes.

Resources to use in an Evaluation Plan

• Evaluation Tools

This site has a range of useful samples e.g. a matrix for planning data collection, protocols, checklists, a questionnaire, a sample report and much more.

• Evaluation Cookbook

Lots of information here set out in the form of recipes (methodologies) to help you set up an evaluation. For example: Checklists, concept maps, likert scales, questionnaires, interviews, focus groups and lots more.