Community Media/CRSC NEPAL/CRPAS

Community Radio Performance Assessment System (CR-PAS)
Manual for the assessors and the community radios on the concept, tool and practical application

Community Radio Support Center/ Nepal Forum of Environmental Journalists Thapathali, Kathmandu with support from Free Voice, The Netherlands.

December 2009

Open Office Version of Final Document

 * Community Radio Performance Assessment System, Published Edition, 2009
 * [[Media:CR-PAS_Manual_FINAL_Dec2008.odt|Community Radio Performance Assessment System, Final Draft 2008. (Please note that the final draft content is also available on this wiki page.]]

Prepared by:

 * Raghu Mainali
 * Yadab Chapagain
 * Bikram Subba

Acknowledgments
Many people have contributed to develop Community Radio Performance Assessment System (CR-PAS) in different ways. The system would not have been in this form without their contribution. The authors wish to thank the following in particular.

'''Special thanks goes to station managers/chair who actively participated in a series of design workshops. They are:'''

Tanka B Thapa (Radio Rupakot, Khotang); Kamala Kandel (Radio Purwanchal, Morang); Jibachha Choudhary (Radio Samad, Siraha); Dilendra Subba (Radio Sumhatlung, Panchthar); Sajan Poudel (Radio Tamor, Taplejung); Ramala Singhak (Radio Menchhyayem, Terathum); Krishna Raj Shrestha (Radio Khadbari, Sankhuwasabha); Dwarika Kafle (Radio Sindhuli, Sindhuli); Jagat Dong (Radio Namobuddha, Kavre); Shivaji Gayak (Radio Jagaran, Rupandehi); Gam B Gurung and Dhan B Gurung (Radio Marsyangdi, Lamjung); and Pushpa Choudhary (Radio Gurubaba, Bardiya). Thanks also goes to resource persons of the design workshop for their contribution. They include Bishnu Puri (National Programme Manager, Decentralized Financing and Development Programme, UNCDF, Nepal); and Bharat Bhoosal and Pratik Bhandari (CRSC/NEFEJ).

Thanks are due to the following for providing invaluable expert input on the draft:

Brigitte Jollov (…. Denmark); Ian Pringle (… Canada); Shane Elson (… Australia); Ashish Sen (President, AMARC Asia Pacific, India); Suman Basnet (Coordinator AMARC Asia-Pacific, Nepal). Sincere thanks to Wijayananda Jayaweera (Division for Communication Development UNESCO, Paris); Steve Buckley (President, AMARC International); Victor Joseph and A

Sincere acknowledgment goes to CRSC/NEFEJ, Nepal; and Free Voice, The Netherlands for availing organizational and financial support.

'''We would also like to thank personalities involved in radio movement in Nepal and NEFEJ members for their encouragement and feedback. They include:'''

Rajendra Dahal (Press Advisor, President of Nepal); Mohan Bista (General Secretary, ACORAB Nepal); Madhu Acharya (Director, Antenna Foundation Nepal); Upendra Aryal (President, Equall Access Nepal); Arjun Dhakal (Member, NEFEJ); Mohan Mainali (Member, NEFEJ); Om Khadka (Executive Director, NEFEJ); Dhruba Basnet (President, NEFEJ); Ghamaraj Luintel (Station Manager, Radio Sagarnatha, Nepal); Tikaram Rai (General Secretary, NEFEJ); Bhairab Risal (Former President NEFEJ); Luxman Uprety (Former President NEFEJ); Sahaj Man Shrestha (Former President NEFEJ); Subod Gautam (Tresurer of NEFEJ) and D.L. Bahandary (CRSC/NEFEJ).

As the CR-PAS is a dynamic tool, which warrants regular improvement, the authors would expect continuous support and feedback from all the personalities and organizations mentioned above. Moreover we expect experience and feedback from the community radio stations, users, national/international practitioners and experts in the improvement process of CR-PAS.

The issues and challenges facing CR movement
Community radio sector in Nepal has come a long way since 1997 when Radio Sagarmatha, the first non-governmental and non-privately owned radio station of the country and also that of South Asia was established in Kathmandu. It took four years for the second one to come up in Lumbini and 10 more years for the number of stations to grow in a substantial fashion. The sector has grown in geometrical proportions in the last couple of years and the number of community radio stations stands at over hundred, which is still growing. The sector’s quantitative growth has given rise to a higher level of information dissemination and has impacted positively on people’s empowerment, promotion of good governance and transparency, and increasing accountability in the public sector in general. The sector has benefitted hugely from the goodwill and support of international organisations. The advent of Radio Sagarmatha was to a large extent also due to the support of UNESCO and the sector continues to receive the Organisation’s support.

The quantitative growth of the sector has been accompanied by the growth in the number of challenges and difficulties as well. Many of these are external. The lack of a clear-cut policy for community radios and the growing number of attacks against free media - also culminating in the form of threats and attacks against community radio stations across the country - are challenges that are externally inflicted.

There are initiatives taken by radio promoters, CRSC and the association of community radio (AMARC, ACoRAB) to distinguish community radios from commercial ones. It is also equally important to distinguish community radios with community characteristics from those who claim to be community radios but do not possess internationally accepted characteristics of a community radio. However, there is no means to differentiate a community radio from commercial one; and a community radio with community spirit from those that claim to be community radio without necessary features.

UNESCO, The Netherlands based mecdia support organisation Free Voice, and the human rights and good governance wing of the Danish International Development Agency-DANIDA are the key supporter to promote communnity radio in Nepal from the very beginning. The establishment of Radio Sagarmatha, the first non-government, non-commercial radio station marked a breakthrough in NEFEJ's struggle to promote community radio. Yet potential community radio broadcasters in Nepal faced a host of challenges in terms of legislative indifference, bureaucratic hurdles, resource constraints and human resources shortage. Once Radio Sagarmatha was well-established, it was natural for NEFEJ to continue to work for promoting community radio in rural areas across the country where such stations are even more essential. It assisted, in more ways than one, the few community radio stations that were in embryonic stage. But to work in a more focused way to help the community radio movement grow, NEFEJ had to have a full-fledged wing. And thus was born Community Radio Support Centre (CRSC) in early 2000.

For community Radio Support Centre (CRSC), the tasks were and are many. With support from FreeVoice Netherlands, has been actively promoting and supporting the growth and strengthening of community radio with help desk, grant fund, technical and management inputs, producers capacity building, reference materials development, advocacy and lobbying, radio knowledge centre etc. CRSC has been working as an epicentre to create a wave of community radios across the country. The main objectives of CRSC are following:


 * To assist those interested in setting up community radio with technical and professional expertise as well as in securing funds for them.
 * To strengthen networking among community radio stations, community radio promoters and independent community radio producers, and to facilitate exchange of radio programmes among them
 * To assist communities to establish community radio in different geographical areas keeping in mind the linguistic, cultural, ethnic and social diversity.
 * To strengthen capabilities of established community radio stations with professional and technical assistance.
 * To organise periodic trainings for community radio managers, producers and technicians.
 * To identify appropriate places, from technical and resources points of view, for encouraging establishment of radio stations.
 * To produce manuals and reference materials in community radio.
 * To undertake community radio audience research/studies.
 * To be active in lobbying and advocacy in order to mould public opinion in favour of making laws that are conducive to the growth and expansion of community radio.
 * To establish and promote contacts with like-minded international organisations for world solidarity in favour of promotion of community radio broadcasting.

After 12 years of the establishment of community radios in Nepal, community radio movement now is in the crossroads. Some radios are moving on the way of politicization, some others towards commercialization, some are NGOisation and some are on the way to real community radio. Because of the ambiguity in the classification of community radios and emerging new trends within the Nepali radio movement, it is facing the problem as to which radio to provide support, which one to give priority and whose request to ignore. And community radios itself do not have a roadmap where to go. With a view to address this concern, it has developed a tool – Community Radio Performance Assessment System (CR-PAS) manual in Nepal. The manual describes the tool and the system of assessment.

Community Radio Performance Assessment System and its Objectives
The Community Radio Performance Assessment System (CR-PAS) developed by CRSC is inspired by the performance based grant system adopted by some countries in the world for providing central government grants to the local authorities, which is also in practice in Nepal.

The system involves ascertaining which community radio to assess, and categorizing the assessed community radios into four groups – Evolving, Progressing, Performing, and Model community radios. CRSC will assess the radios that have renewed their license, have declared to be community radios, and are willing to be assessed as the minimum conditions for performance assessment. For categorizing the radios into four groups, a set of performance indicators will be used and the performance will be scored with 100 as the full score.

The community radios’ performance indicators are drawn from international good practices for a community radio, good governance measures, and the context of Nepal. Besides, these the indicators and system is developed jointly by a team comprising professionals at CRSC, experts in performance based grant system, management experts, and 11 leading station managers of CRs in Nepal, through a series of workshops, which enhances the legitimacy of the tool.

The CR-PAS also offers an objective basis for CRSC to identify the capacity gaps of community radios, and select, prioritize and decide the community radio to provide its support. Besides this, it also gives a roadmap to the CRs as well – they can see the performance areas where they should focus to improve as true and effective community radio. The state and the public institution also benefit from the assessment – they can decide which community radio to work with, to channel their programs and resources through, and to provide state incentives. The public authority can also use this system as a part of their general monitoring and evaluation system

.

Moreover, the system is expected to contribute to policy formation process with regard to defining the community radios.

Objectives
The performance measurement tool gives a category to a radio station, which may impact on the donors support, CRSC-FreeVoice support, and the way a radio will be viewed by the public and authorities. Therefore such an assessment should be done with no preoccupation and it should be impartial, objective, and uniformly applied to all. Moreover, since the result of the assessment is also for public consumption, the process, indicators, scoring system, and the result should be made transparent to all concerned. The manual also provides guideline to the assessors and make the measurement simpler and objective.

The manual is prepared to ensure these objectives.

Note about CR-PAS and the manual
The community radio performance assessment system and the tool are simple but have a multiple use as mentioned above. This therefore has potential to attract attention of different agencies within the country and abroad. This is welcome as an intention of the manual is to make the system and the tool popularly used. We think that it is better used with an understanding of the basic considerations and limitations of such a system and tool.

The performance of a radio is measured in terms of the scores obtained by a radio on the indicators in different composites. Because of practicality of application it was necessary to choose few composites and indicators as far as possible, and at the same time it was also essential to strike a balance covering the complexity of the environment a community radio operates. In other words, the composites and indicators should not be taken as rigid and absolute.

The composites and the indicators are therefore a result of optimizing balance, adequacy, and simplicity. Hence, though we gave due respect to the comments and suggestions obtained to improve the manual, but could not accommodate all. There are several reasons for not incorporating all the relevant suggestions. For example we scored low for impact assessment of its programs by a radio because we observed that it would be too demanding and costly an indicator if we asks this from radios in Nepal at present. Moreover, the tool is designed to guide the radios to improve; it should not be taken as an evaluation tool. Therefore, we did not include indicators such as audience feedback. However, it gives indication as to the strong and weak aspects to interested evaluators. For the same reason the manual retains some indicators even though they are static (once adopted no need to further work on it). Such indicators become obsolete after an assessment and improvement exercise, so should not be included in such a tool where measurement is done in a certain intervals. However, we have decided to retain them because they were important. We intend to drop them later when the situation in general improves.

In short, the manual provides a tool to measure performance based on objective criteria, but realizes that the tool is not complete. Neither is it an end. This is simply a beginning; it is envisaged that a new set of composites and indicators will be continuously developed with experience and changing context. The tool rather gives a broad frame where the content can be worked out according to the country context, time, objective in mind, vision for community radio, and general situation in which the radios operate.

Performance Assessment System and Indicators
The tool is the core of the CR-PAS. Before describing the tool itself, it is deemed important to recon the major considerations taken into account while designing the system.

Considerations of the performance assessment system
The CR-PAS is designed taking into consideration of the following:

Acceptance and credibility 

The system is developed with intense consultation with the practitioners, experts and other stakeholders. The good practices, norms, and practical reality are included to enhance acceptance. The assessment will be done with independent professionals to ensure credibility.

Practicality and simplicity 

The system and indicators are simple and easy to measure. The scoring system is objective to make it further simpler and understandable to all interested. The assessment can be done in a couple of days in a radio to make the system practical to carry out assessment radios across the country. As far as possible the system is designed in such a way that the assessment process is cost effective.

Comprehensive and focused

The system attempts to cover all performance areas of the community radios. At the same time it focuses on a few, but the key, indicators that are critical to development of a true community radio. Performance measures are realistic, achievable and objectively verifiable, i.e. clearly defined, but still sufficiently demanding to promote improvements.

Weightage based on importance

Scoring system is designed in such a way that the more important areas of performance and the more critical indicators are allocated higher scores than others. The importance is assigned by the people managing and promoting community radio themselves.

Positive reinforcement

The system is designed to provide incentive to better performers, and to show improvement areas for those who could not do well in the assessment.

Measurement of accountability

The system attempts to measure those aspects that are directly under the control of the radio station, and those that are directly associated to the accountability of the radios. Therefore there are more of process indicators for measurement – the areas of impact are intentionally left out since the impacts generally are not attributable to a single entity.

The community radio performance measurement tool
The indicators are identified in seven composites that comprise the various key areas of performance. A total of 100 score is divided among the composites, which varies across the composites based on their importance. Total minimum score for the radio is 35 out of 100 scores, and a successful radio must obtain the specified minimum scores separately in each composite. The category of the radio is ascertained based on the total score obtained (with minimum score in each composite). The tool is described in the following section.

Composites and weightage
The table below gives the total score to different composites and minimum required score for each composite.

Categorization of the radios
The community radios will be categorized based on the following scores each secures in the performance assessment system.

Note:


 * If a radio scores high in total but fails to secure minimum scores in one or two clusters will be grouped as the immediately lower category. For example if a radio obtains 65 as total score but fails get minimum score in a cluster will be categorized as “progressing”.
 * If a radio secures minimum 35 marks but fails in one or some clusters then it will still be groped in “evolving” category. 

Indicators and corresponding score
The following indicators are identified in each composite.

Composite 1: Participation and ownership (Total score 20)

Composite 2: Radio governance (Total score 15)

Composite 3: Radio programs (Total score 25)

Composite 4: Resource structure & resource management (Total score 15)

Composite 5: Radio station management (Total score 10)

Composite 6: Financial management (Total score 10)

Composite 7: Networking (Total score 5)

The performance assessment guideline with explanation of the indicators, source of information for assessment, means of verification and the basis of scoring is provided in following section.