Quality Assurance in Multimedia Learning Materials

In order to address quality concerns for the fast burgeoning development of multimedia learning materials, the Commonwealth Educational Media Centre for Asia (CEMCA) has spearheaded an initiative to develop a framework for quality assurance in which is seconded by the Ministry of Higher Education, Malaysia. Over the past one year, two core groups, one each in India and Malaysia, comprising academics, practitioners and industry have been grappling with developing a framework. Periodically, through various stages of development, wider consultations have also been held in both countries. The two groups met in Kuala Lumpur on November 1, 2008 and finalised the version 1.0 of the draft.

Some Indian team members at Batu caves in KL.

The Malaysian group at the meeting, Nov.1 2008.

We would like to share the draft with a much larger body of stakeholders across the Commonwealth before releasing it as a formal document. So we have made it available on the CEMCA website as well as in PDF format here.

The ADDIE model of Instructional Design was followed for the Quality Framework. The framework consists of a table consisting of Activity, Input and/or Processes, Outcomes and Quality Indicators under the following heads: Analysis, Design, Development, Implementation and Evaluation.

In the present Framework an attempt has been made to identify the core components / key ideas that reflect Quality with respect to each Activity and sub-activity. The specific approach adopted when developing the Quality Indicators (QI) has been as follows:

• To identify all the major issues that have a bearing on the perceived Quality of the MLM at each stage of its development.

• To prepare an elaborate, though not exhaustive, list of pointers of Quality, taking care to avoid redundancy as well as duplication.

• To state the Quality Indicators using simple, unambiguous language that captures the essence of what reflects quality.
• To develop a set of Indicators that not only help to identify whether Quality is present or absent, but also pave the way for assessing the extent or degree to which a certain Quality Indicator is present. (i.e. to facilitate an understanding not merely of whether something has been done but how well it has been done).

Quality Indicators

Based on the Quality Indicators finalised after intensive deliberations and scrutiny as to whether each Indicator is a needed, relevant and critical component of Quality, the next step taken was to develop a scale of Assessment for each. While recognizing the fact that a good indicator of Quality need not always be quantitative, in order to provide a readily usable, uniform format that is consonant with the approach adopted by most Quality Assurance agencies and one that permits objective comparisons between different MLMs, a five-point scale has been developed for each Quality Indicator. The lowest end of this Scale (Level 1) corresponds with the Verbal descriptor, ‘Poor’, and the highest point (level 5) represents the ‘Excellent’ level. The five point scale used for each QI and the progression implied from one level to the next is as given:

1. Poor ---------- generally representing absence or non-existence or no consideration given to a certain QI.

1. Average ------- indicating few components, partial presence or marginal consideration given to a certain QI

1. Good -----------indicating presence of or consideration given to many components of the QI

1. Very Good-----suggesting presence of or consideration given to most (almost all) of the important components of the QI

1. Excellent ------ indicating presence of / consideration to all the components of the QI PLUS some value addition (e.g. facilitating / providing direction to the next steps in the development of MLM.

While in the present form, no weightages have been assigned to the Quality Indicators, the approach for the future could be towards assigning differential weight ages to the different QIs depending on their relative importance in the overall development of MLM, and arriving at activity wise weighted numerical scores. The Quality indicators along with assessment parameters are given in the following links:

1. Analysis
2. Design
3. Development
4. Implementation
5. Evaluation

Quality Assurance of Products

We have also developed some product related indicators that are available in the draft form here.

We invite your feedback, comments and suggestions by February 15, 2009. You may :

• Input directly on the discussion page attached to this wikieducator page:
• Send us an email to RSreedher@col.org or rvemraju@col.org or savisingh@and.du.ac.in
• Send regular mail:
  

Commonwealth Educational Media Centre for Asia
8/4, Sarvapriya Vihar, New Delhi 110016

Please do include your name and affiliation if you would like to be acknowledged as a contributor in this process.