Impact Assessment of BDS and Training Programmes
Dear Jealous,
Thanks for this interesting topic again.
Apart from The DCED recommended universal indicators on private sector development. Generally the following key indicators are common for the propose of the BDS impact assessment. Most of them are already reflected in your discussion question above.
• Number of BDS providers recruited,
• Number of support actions identified
• Number of trainers and facilitators trained,
• Number of courses held,
• Number of trainees participating in the training
• Number of transactions and amount of the transactions
• Type of BDS services provided/offered
Methodology: In our case before any interventions, we use to develop result chain logic/intervention logic (with clear indicators on Inputs, outputs ,outcomes and impacts).Control group mechanism is one of the best approach to compare the results
Dear Ekanath,
Good evening and thank you for expanding on these. And the reference to the DCED guidelines, which is useful for everyone participating in the discussions. I urge everyone to google their website. In terms of the indicators, we can have another entire discussion there, so that we are able to go beyond the output / activity level indicators to more outcome oriented ones. Those that measure the results of what has been done (outputs), for instance number of new businesses started, increased sales and incomes of the business oweners, number of SMEs and amount of credit accessed, number of new jobs. these will better show results.
I am happy you share the notion that control growup methodology is the best. I agree as it helps us to answer to the question of attribution. In other words, did the intervention contributed to the results? Would the same result not have been achieved without your intervention. This is really key. We need to exchange more on this. Jealous
Dear Jealous
Thanks for your elaborative response. Yes impact assessment on BDS interventions is one of the most debatable issues. Another issue is time frame. According to the DCED standards, the standard time frame for impact assessment is 2 years . another critical issue is attribution which you have raised / questioned in your respond. I totally agree with you that actual result attribution is another challenge. Without our particular interventions there might be many other interventions and supports, therefore while attributing the results/ impacts we have to consider the element of “coping in” and “crowding in “aspects. As per the DCED guideline/ standards, if it is possible we need to distinguish the direct and indirect results in our result chain logic.
Dear all, Please visit the webpage www.enterprise-development.org for more DCED guideline and result measurement standards.
With best regards,
Ekanath
Thank you very much for you response
For me indicators to measure impact should be well framed. They are supposed to depict the results that one needs to achieve. As i pointed out the key word is what did we achieve and not what did we produce in terms of products or services, this is a very key guiding principle. Now what do we need to monitor as in indicators we can frame for us to ensure that we do not miss our mandate. For me if we are to look at number of courses held, number of transactions transacted and number of BDS providers recruited, then we are going to miss the point. These indicators are giving as output and they are therefore only used for monitoring purposes. Impact is a product of evaluation and that is more inclined to outcomes. If we look at number of trained entrepreneurs that are running there businesses, Owning companies ,Number of Enterprise using the market approach, number of people trained who are able .....e.t.c These type of will help us to know what we have achieved in the light of impact,9reducing poverty,creation employment among other social economic dynamics.
On the Other hand logical frame approach is a very important tool that one can use to monitor and evaluate their programs it is been used extensively noways and it is a donor requirement in most cases so forks it is only better to know it and use it. Logical framework will tell you what risk assumptions to expect,frequency of data collection , under which objective is that activity being carried out, Goal , output and outcome expected. It is simply a beautify monitoring and evaluating tool as it even give provision for the person responsible and the cost involved. I should however, make a mention here that care should be exercises especially when crafting these two tools, indicators and logical framework as it may not be able to detect any deviation from the normal implementation of a program. You may expend money on programs that does not address the objective of that program. However, a robust monitoring mechanism will within no time detect a problem at any given time and help you to make timely strategic management intervention. regards
Dear Isaac,
Thank you very much for your respond. I really appreciate your ideas that we should not focus our impact assessment around outputs only and we should design more impact focused interventions.
Yes I do agree with you about the importance of Logical framework approach which is more elaborative frame work for result measurements. But I do prefer to use result chain logic or intervention logic . The result chain logic/intervention logics are much more easy frameworks to connect inputs- outputs - outcome and impacts logics .
With best regards,
Ekanath
Dear Ekanath,
Thank you very much for the your response and clarification on what you use and especially the one which is seemingly easier for you to track. Impact measurement is just in a way like any other activity that a program may be mandated to implement. The key thing is for one to own the process and be able to use it effectively,enable you to use your result efficiently and ultimately ensures your organization's project sustainability. This is only also again possible if we take keen interest in formulating frameworks that we can use effectively. I have been working as a consultant on evaluating management effectiveness of programs. At the project design stage good and robust frameworks are adopted some are even way above the organization M&E capacity needs but because people what to please donor they adopt and later labor to adapt. This kind of frameworks beautiful as they may look they do not ultimately help the organization to attain its desired goal. This is waste of time and money. Designing a simple one we understand and be able to offer a clear cut way to tracking progress in results, it is actual thousand times progress than adopting one that one can not use. In life academic exercise do not apply only what one is comfortable with. Sorry Ekinath, this is not to say yours is simple iam just trying to stress the need a for simple and one comfortable models. Good and robust M&E system calls for the adhering to the adaptive management principles, able to monitor results, adapt, analyst ,learn ,communicate and adapt best practices. All these are taken care of in my COPIPO framework for project assessment and impact diagnosing. Regards
