Quality Assurance/QA Toolkit

QA Toolkit for Open and Distance Learning for Non-Formal Education

Source: Quality Assurance Toolkit for Open and Distance Non-Formal Education, by Colin Latchem, Commonwealth of Learning, 2012

Getting started

Stick to the facts and make sure there is evidence to support all your judgements and statements. Otherwise, the quality of your (educational) QA will come into question. (p. 82)

But rather than starting by using these inputs as predictors of quality, we suggest you assess the outputs, outcomes and impacts first, and then refer back to the quality of the inputs to see where, why and how things went wrong or right. Traditionally, measures of quality were exclusively or largely based upon the inputs because they were easy to observe, measure or compare, and believed to be highly correlated with educational success.  (p. 82) But essentially, QA is a matter of, as James Ling said, “Don’t tell me how hard you work. Tell me how much you get done,” or as Peter Drucker observed, “Quality in a service or product is not what you put into it. It’s what the customer gets out of it.” (p. 84)

 

QA Framework for assessing outputs (p. 82)

Outputs	Performance Indicators
In the learners	* The Program attracted the numbers and types of participants planned for. The learners’ attendance, retention, motivation and participation levels were high. The Program made use of the learners’ tacit knowledge as well as their explicit knowledge. The knowledge, skills and attitudes aimed for were reflected in the learners’ behaviours during the Program.
In the learning provision	* The planners and providers worked with the targeted groups to assess their knowledge needs, abilities, culture and circumstances. The needs assessment was sound and accurate. Strong linkages/partnerships were achieved with educational/training providers and other agencies and groups interested in the specific development. These linkages/partnerships helped to engage the communities, recruit learners/trainees and provide the learning activities, learning materials and support services. The managers, teachers/trainers, facilitators/mentors, technical assistants and other support personnel were recruited with care. The managers, teachers/trainers, facilitators/mentors, technical and other support personnel were appropriately trained in the systems, methods and technologies to be used in the Program. The aims and potential benefits of the Program were widely and effectively marketed and promoted. There was sound formative and summative evaluation of the Program. The objectives of the education/training were realized. The Program ran to schedule. The teachers/trainers and facilitators/mentors performed well. The course, course materials, classes, practical training activities, etc. were well-conceived and well-implemented. The choice of technology, infrastructure and means of delivery were appropriate. The OER were appropriate or appropriately localized and customized. The distribution of the courseware and learning support services were well managed. The management and administrative support was sound. Funding matters were well and transparently managed. The technical support was well managed and operated. The Program was based upon a sound business plan.
Within the community	* The planners and providers involved the community leaders and other stakeholders who understood the needs, potential and limitations, as well as the most appropriate means of communicating and working with the communities. Local “champions” were involved to achieve maximum interest and participation. The learners/community were involved in the design, implementation, management and evaluation of the Program. Sound partnerships were established with local educational institutions, NGOs, businesses, media channels, etc.

QA Framework for assessing outcomes (p. 83)

Outcomes	Performance Indicators
In the learners	* The knowledge, skills and attitudes aimed for were reflected in the learners’ behaviours following the Program. The applications of the new learning were shown to be effective and beneficial to the learners and/or community. The Program empowered the participants and engendered a new sense of self-worth and enterprise. The participants continued functioning as a community of practice or virtual community. The participants undertook similar Programs on their own initiative.
In the learning provision	* The new approaches were shown to be cost-effective and sustainable. The Program acted as a catalyst for further developments. The Program attracted other partners and stakeholders.
In the community	* The participants recommended the Program to other community groups. Other community groups requested or adopted the Program.

QA Framework for assessing impacts (p. 83)

Impacts	Performance Indicators
In the learners	* Continuing motivation, persistence and building of skills and knowledge. More critical reflection. More experiential and collaborative learning. Learning communities engaging in lifelong learning. Increased creativity and adaptability.
In the learning provision	* The new approaches continued to be shown as cost-effective and sustainable. The Program continued to act as a catalyst for further developments. The Program continued to attract other partners and stakeholders.
In the community	* Improved attitudes, knowledge and skills. More active citizenship and personal development. Greater capacity to manage uncertainty. Increased community enterprise and capability. More economic resilience and self-reliance. Increased enterprise and competitiveness. Greater productivity. Improved employment or self-employment prospects. Learning pathways into formal education for those wishing to further their education. Enhanced social inclusion. Learning to live together and with other communities. Conflict resolution. Communication across and within cultures and sub-cultures. Improved healthcare and child care. Increased opportunities for women. Less pollution and environmental degradation. Greater food security. Eco-enterprises, eco-preneurship and eco-jobs. Less corruption. More micro-credit Programs.

QA Framework for assessing inputs (p. 84)

 

Inputs	Performance Indicators
Policy and planning	* The provider has a clear sense of purpose and direction, based on national priorities / community needs and the quality demands of cost-effective educational provision. There are both a rationale for and relevant systems for the use of distance education to achieve the Program’s aims for the targeted learners. The provider has consulted with all of the stakeholders to ensure that the processes and outcomes accord with their needs and expectations. The provider has ensured that learner support mechanisms are in place and contingencies have been planned for. The provider has developed a QA framework and implemented a QA procedure. There are policies to ensure that the physically challenged members of the community, women, and other disadvantaged or marginalized persons have equal access to the Program. Where appropriate, learning pathways and/or credit/certification have been planned for.
The learners	* There are appropriate publicity, recruitment and enrolment procedures for the learners. There is up-to-date and relevant information about the learners, their ages and gender, their needs, their prior learning and experience, and their circumstances. There are records of the learners, their contact details and their performance on the Program. There are incentives for the learners to study.
Management and administration	* There is effective, transparent and democratic management of the Program, human resources and communications. The Program is supported by efficient administrative systems, both at the centre and in the community. There are clear lines of accountability to all of the stakeholders. The staff, learners, community and external stakeholders are represented in the governance of the Program. There are efficient and effective systems for communication with the stakeholders. Inquiries, complaints and problems are dealt with quickly, efficiently and effectively. There are centres and systems in place for remote learners.
Staffing	* The teachers, trainers, facilitators, mentors, and the local managers and administrators have been carefully selected and appropriately trained for the Program. The teachers, trainers, facilitators, mentors, and the local managers and administrators are appropriately recognized and rewarded for their contributions. There are records of all staff and their performance on the Program.
Technology and infrastructure	* The technology and infrastructure are well-provided for. The technological and infrastructure provision are inclusive. There is an appropriate mix of technologies and technology-based delivery methods. The equipment and facilities are well managed and are maintained and secured against theft, misuse or damage. Staff and learners are trained in the use of the equipment, facilities, and communication and information systems. There are emergency procedures in the event that the primary communication fails.
Funding and budgeting	* The funding and sources of funding are appropriate and reliable. Appropriate budgetary processes and financial procedures are in place, known and followed. Accounting and auditing systems are in place to compare goals, budgets, outcomes and impacts. The costs of study are affordable to the community and learners.
Program development	* The Program is relevant to the national, or community, or individual needs. The Program is appropriately developed or adapted to the learner profile. Relevant stakeholders are involved in conceptualizing the Program. The Program is flexible, accessible and responsive to changing needs and circumstances. The Program is culturally appropriate. The Program combines an appropriate mix of teaching, group study, independent study and practical work. The course, materials, learner support and assessment are well planned and well-implemented. The Program is evaluated regularly. There is a reliable system of costing Program and course development and evaluation. The location, timing and pace of the Program are appropriate to the learners and their circumstances.
Teaching and learning	* The teachers/trainers are committed to quality teaching. The teaching and learning is learner-centred. The teaching and learning takes account of the learners’ tacit or informal learning and prior experience. The teaching and learning makes appropriate use of up-to-date theory and practice, as well as empirical and indigenous knowledge. There are appropriate time/work patterns in the teaching and learning provision. There are appropriate workload factors in the teaching and learning provision. The forms of teaching and learning used are culturally appropriate and inclusive. Feedback, reinforcement and reward are provided. The quality of the teaching and learning is judged by peer review or feedback from the learners.