Ipyet/Monitoring and Evaluation of Training Programmes

Work in progress, expect frequent changes. Help and feedback is welcome. See discussion page.

Module 5.2: Monitoring and Evaluation of Training Programmes
A discussion paper by
Jealous Chirove

Part 1: Introduction

This session will discuss monitoring and evaluation of training programmes. This is an important matter, which can enable us to assess the impact and reach of or training interventions to provide information into the next planning loop. In spite of this, M & E is often neglected, not allocated adequate resources, or at best attended as an after-thought when interventions have been implemented, making it difficult to effectively assess and answer the key questions of attribution. The main question that is always asked is which one is the best method in evaluation? When should evaluation be done? How can we monitor training programmes? The key question of attribution has to be answered. Most important is how you can develop and institute a monitoring and evaluation mechanism before or at the start of your interventions.

Objectives

Session Objectives The objectives of the session are; To enable participants describe Monitoring and Evaluation and how it can be applied to training programmes for youth entrepreneurship To introduce various Monitoring and Evaluation methodologies To discuss the steps involved in monitoring and evaluating training programmes

Part 2: What is Monitoring and Evaluation?

Monitoring is the routine tracking and reporting of high priority information about a programme or project, its inputs and intended results. Monitoring information has to be used at all levels of the system for self-assessment and tracking the progress and decision making. It is therefore important for everyone involved to be clear about which indicators are being monitored as well as the information to be collected and how.

Evaluation is an assessment, as systematic and impartial as possible, or project, programme, strategy, policy, theme, sector, operational institutional performance. Evaluation aims at determining the relevance or appropriateness, effectiveness, efficiency, impact and sustainability of the interventions and contributions to the organizations. The evaluation process and methods should be decided during the initial planning process. This requires prior planning and may require the use of external resource persons / experts. Evaluation should be based on a set of indicators developed during the initial planning process.

Reflection

Reflection Question: Does your project have a Monitoring and Evaluation plan? Did you develop a set of output, outcome and impact indicators at the start of the project? Why is important to have these done at the start of the project?

Why is Monitoring and Evaluation Important?

Monitoring and Evaluation is important for a number of reasons. These include;

1. To enable the project to be accountable to national stakeholder partners and beneficiaries, your own organization, and donors or funding partners. In the case of training, you need for instance to give numbers of how many youths have been trained in business skills, by gender, age and other important variables such as business characteristics.
2. Monitoring and evaluation facilitates organizational learning and continuous improvement of your organizations. This important as the monitoring and evaluation function bring up lessons learnt and success stories that can draw up important pointers for the next level project planning cycle. Through M & E you can be able to know the outreach and gender composition of the target group reached and this can allow you to assess whether you are in line to meet the set targets. If not, you can then take appropriate action to correct that.
3. Monitoring and evaluation also increases the visibility of the contribution your organization is making to sustainable improvements in your operational areas, sectors and the country at large. When you give out numbers of young people reached through your programmes, young people provided with business skills and those who have started their own small businesses, your organizations starts to become relevant and creates impact. This information should easily be available through your M&E system which should be updated and produce standard reports periodically.
4. Monitoring and evaluation is today an integral means of managing for results. It is part of the project cycle. It is no longer an option but a requirement. Donors and funding partners are under pressure from their Governments and Parliament to show the results of their support. They therefore expect the implementing organizations and projects to deliver and show these results. The monitoring and evaluation helps you to show the results and impact.

Important! Many programme staff think Monitoring and Evaluation is a burden on their work. They look at the M & E function as either time consuming, expensive or maybe an unnecessary activity. That view needs to change as it not correct. It is only through M & E that you can know whether you are doing the right things in according to the set programme. It is only through M & E that you can show the outreach and impact of your interventions. That way, you can know which of your interventions are most effective and which ones can be scaled up.

Part 3: Monitoring and Evaluation in Training

The following 5 principles should be applied in all youth training programmes to ensure a robust monitoring and evaluation function;

1. All youth training programmes should have a Monitoring and Evaluation plan. If you do not have one yet, go and work on it.
2. The M&E plan should be developed during the initial project design phase and refined during start- up phase. This should be done in a stakeholder planning meeting
3. Adequate resources should be allocated for M&E. As a rule of thumb, a minimum 5% of the total allocation budget of the project, of which 2% for independent evaluation.
4. The M&E plan should have
   • Indicators
   • Baselines, Targets and milestones
   • Means of Verification, Frequency and responsibility to collect the information
5. All stakeholders should agree on the M&E plan before its implementation.

Tips in Monitoring and Evaluation of Training Programmes

In training, monitoring and evaluation should be done before, during and after the training event.

Before the training event, baseline data for each of the participants needs to be collected. It is important to know the situation before your intervention. This baseline data should be kept as part of the M & E system.

During training, you need to collect information about how the training programme is going, what feedback is coming from the participants and assess the whole training event at the end of the training. You may decide a Daily feedback / Evaluation form to assess the views of participants at the end of each day. This is important to also help you improve on all aspects of the training event. At the end of the training, you may want to use an End of training evaluation form to assess the overall training event and how the participants feel about it.

Six to twelve months after the training, you may want to conduct an impact assessment to assess the overall change in the participant situation following the training. The period following the event should be long enough to allow for any impact to take place, but not too long enough as many other things may happen in between which can affect the participant and bring to question the ‘attribution’ question.

Part 4: Monitoring and Evaluation Indicators

Indicators are a key component of the whole monitoring and evaluation system. An indicator is in simple terms a variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention. The indicator is therefore a tool to measure evidence of progress towards achieving a result or that a result has been achieved. In any programme or project, indicators depend on the outcomes / objectives. While results tell us what the programme / project plans to achieve, indicators tell us how to measure and ascertain if those/outcomes are realized or not.

Types of Indicators

There are basically two types of indicators;

1. Quantitative (numeric) indicators, which can be expressed as a percentage, number, proportion, etc. An example is the percentage of young women from rural areas who received business skills training
2. Qualitative (non-numerical) indicators, which are used to assess the level, degree, exact scope, of a factor. An example is the level of satisfaction of the trainees by a training programme on a scale of 1 to 4, where a 4 is highly satisfactory and 1 is not satisfied at all.

Indicators for different level of results

Indicators should be formulated for the level of results (impacts, outcomes, outputs) and also for programming processes:

1. Impact indicators. Impact indicators describe the changes in people’s lives and development conditions at global, regional and national levels. Impact refers to the realization of sustained, positive changes in the life, dignity and wellbeing of the people e.g., level of poverty.

(ii) Outcome indicators for outcome level results / objectives. Outcome indicators assess progress against specified outcomes / objectives. They help to verify that the intended positive changes in the development situation have actually taken place. Outcome indicators are designed within the results framework (log frame) of global, regional and country programmes (project). Outcomes indicators are most often useful to the ILO and their partners working on the specific outcome.

2. Output indicators: These are the detailed expressions of the programme / project results for the project is responsible. Output level results are the goods, services and deliverables produced, e.g., number of business plans submitted for funding after training.

Assignment

Exercise 1 Can you write down 2 indicators each at output, outcome and impact levels for a youth training programme. The indicators would show what is being measured at the three different levels

Part 5: Baselines in Monitoring and Evaluation

A baseline is a description of the situation at, or prior to, the start of the (training) intervention, against which progress can be assessed or comparisons can be made. The first time that data is collected on the indicator a baseline is established. Normally, baseline information should be collected before we start implementing the programme.

Targets and Milestones

A target is the level of change / achievement expected per organisations commitment by the end of the project or a programme. A target states how much process is anticipated towards outcome. Targets are the baseline measurement plus the amount of improvement one hopes to realize.

Milestones are normally used for projects with longer timeframe. In simple terms, they are targets are divided into time-bound increments called milestones. Milestones define what the programme aims to achieve by certain points in time (e.g., end of each year). They tell us what whether we are advancing in the right direction at the right pace to reach that destination as planned or whether change is needed.

Part 6: Use of M&E Information

It is important to note that evaluation does not end with the evaluation report completed. One of the aims of the monitoring and evaluation is to translate its finding and recommendations into action. The value of M&E is determined by the degree to which the information is used by intended decision makers, constituents and programme and project staff. The knowledge gained from M&E is at the core of organizational learning process.

Monitoring and evaluation information can be used for;

1. Discussing and reviewing monitoring results during periodic staff meeting. M & E results and findings impact on the work of everyone in the organization. It is therefore important to present such reports and results for all staff to discuss the recommendations.
2. Presentation of M&E findings in annual reviews that involves programme partners. Your programme partners are some of the most important stakeholders who need the M & E results.
3. Sharing evaluation reports and good practices with other peers, projects and the outside audience through established mechanisms. Peer learning is one of the more effective tools in project design and implementation. Sharing the results of what works helps in guiding the design of other programmes in future or re-orienting current programmes.
4. Implementation of evaluation recommendation using the management response. Management has to take an active interest in the M & E results. The implementation of recommendations will only be made easier and smoother if management are aware of the issues and committed to do it.

Discussion

Discussion Question 1 Which methods have you applied in your project’s monitoring function? Which methods have you applied in evaluation? What have been your experiences in using the methods?

Web Resources

More information For more information please visit the following websites; www.ilosiyb.co.zw www.yen.org