PCF5:‘Wiki standards’ and qualifications frameworks

The Pan-Commonwealth Forum on Open Learning Wiki

A collaborative space to draft and talk about contributions to the Fifth Pan-Commonwealth Forum (PCF5)

Home | About |  Governance, Conflict & Social Justice |  Health | Livelihoods |  Children & Youth |  Cross-Cutting Themes

‘Wiki standards’ and qualifications frameworks

Paul West, Commonwealth of Learning

James Keevy, South African Qualifications Authority

Abstract

How radical is it to consider that any member of the public can post a qualification standard on a website, another can edit the same standard and anyone can update it on a regular basis? Even more radically, can such an approach be accommodated in national-, transnational- or regional qualifications frameworks? In our paper we try to open this debate to a broader audience by taking two opposite positions: Paul, with a background in the Wiki environment, argues that the development and maintenance of standards can be supported through ongoing general consensus, much like a WikiPedia article, and that daily changes to such standards simply reflect the natural daily changes in the world. James, with a background in qualifications framework development, argues that standards can only be developed by expert practitioners located in specific communities of practice.

Introduction

We prepared this paper at the conclusion of the collaboration between the South African Qualifications Authority (SAQA) and the Commonwealth of Learning (COL) to develop a concept document for a ‘transnational qualifications framework’. This collaboration, which took place between August 2007 and May 2008, focused on the development of common quality assurance mechanisms for courses offered within the context of the Virtual University for Small States of the Commonwealth (VUSSC). It was particularly the meeting of the world of qualifications frameworks, strongly biased towards systematic and expert-driven approaches to qualifications development, and the world of open and dynamic approaches to course development that prompted us to open this debate to a broader audience at this conference.

As noted in the concept document (COL & SAQA 2008:16), the idea of a VUSSC was first mooted at the 2000 triennial meeting of Commonwealth Ministers of Education in Canada. Progress has subsequently been made on a number of fronts, overseen by the Commonwealth of Learning. At present 29 small Commonwealth states are participating in the VUSSC initiative .

In a parallel development, SAQA has overseen the development and implementation of the South African National Qualification Framework since 1995. SAQA has also been involved in an investigation into teacher qualifications and professional status in the Commonwealth (SAQA & Commonwealth Secretariat, 2006), as well as a number of related projects focusing on qualifications framework development in the Southern African Development Community (SADC 2005, SADC 2007), the Seychelles (Domingue, Julie, Keevy & Mphuthing 2007) and more recently, also in East Africa (Ministry of Education of the Federal Democratic of Republic of Ethiopia 2008).

Following a request from the participating VUSSC countries in March 2007 to create mechanisms to support the accreditation of qualifications and transfer of credits between countries, COL contracted SAQA to analyse existing qualifications frameworks and find commonalties between them, with a view to using these as a basis for the development of a qualifications framework for VUSSC qualifications across member countries – a transnational qualifications framework. A concept document was prepared by SAQA and discussed with senior officials from 20 countries in February 2008 in Singapore. The document was subsequently revised and will now form the basis of discussions with VUSSC interlocutors, after which further actions will be considered.

In this paper we reflect on how the worlds of open and dynamic standards (represented by COL and the VUSSC initiative) and qualifications framework development (in this context represented by SAQA) have collided. We consider the different approaches to qualifications development that have (e)merged as a result of the interaction, and present this as a case study to encourage further critical deliberations that can inform future approaches to qualifications development.

The open and dynamic approach to standards

The sharing of course materials is becoming more commonplace, and is a founding principle of the VUSSC. Although the free sharing of learning content has been around for decades, UNESCO introduced the notion of “open educational resources” and it received more attention when the William and Flora Hewlett Foundation provided financial support to the Massachusetts Institute of Technology and other institutions to prepare and publish all their classroom notes on the Internet. The VUSSC is taking a similar approach by encouraging small states to share content developed by VUSSC teams. To make materials more useful and relevant, the teams consider the target audiences, the national needs of countries, and the appropriate level.

Matching desired outcomes in multiple countries is a significant challenge. We may assume that basic accounting in one country is the same in another country, but is it? When country A publishes what it deems an appropriate set of standards, and country B believes the standards do not fit the bill, who arbitrates the difference of opinion? Where different countries establish different systems – for example, if one country decides on six levels of qualifications and another decides on eight levels – how does one reconcile the differences? It seems sensible to have an agreed process in place to address these challenges when developing VUSSC standards.

Traditionally, governments have taken on the responsibility for recognising national, regional and even transnational qualifications, often within partnerships with industries that in turn employ the learners who emerge from these systems. Institutions, especially universities, have been known to defend their right to autonomy, believing that no one should question their right to assess the quality of their own programmes. However, universities are beginning to evolve their thinking in this area – some understand that their learners end up in the private sector and that they need to listen to their clients, or risk having the private sector set up its own specialised institutions. This has been quite prevalent in the computer industry, which is notoriously fast-moving. Within this changing environment, universities and other institutions need to be full partners in qualifications framework and quality assurance processes.

More recently, a new group of people expecting to have a stake is emerging – the general public. A member of the public who uploads a video to YouTube, who maintains a blog on her daily activities and who sees herself as a competent maintainer of a dozen encyclopaedia pages on WikiPedia, should also be able to help set the standard. There is a view that standards should be posted on a Wiki site and left to anyone to update, much like a WikiPedia article; that these will find a standard through general consensus; and that the daily changes to the standards simply reflect the natural daily changes in the world. While the inclusion of the general public in such “open” standards development is still to be debated more rigorously, it does emphasise the non-conventional route presented within the context of the VUSSC and, importantly, the possibility of an equally non-conventional approach to the development of the proposed transnational qualifications framework.

Standards are usually fixed goalposts that remain constant over time, though approaches to them may be diverse and vary over time. Inflexibility is both a strength and a weakness. One of the challenges for quality assurance is the tension between the need for, on the one hand, clear non-negotiable standards, stable structures and consistent processes, and on the other for flexibility and the ability to respond and adapt to a continually changing external environment. Nevertheless, performance information from the internal environment which regularly triggers continuous improvement effort, involving critical review of, and adjustments to, the standards, structures and processes, needs to be maintained.

Standards enable almost everything we touch – construction, communication, transportation. Standards apply to hammering a nail into a piece of wood as much as to connecting via the Internet or assessing the level of a unit of learning. Standards may be created by a major supplier and by choices made by the public. In its day, the VHS video recorder became the standard and the Betamax video recorder became almost extinct. This was not a choice by a standards-setting body, but one driven by consumer choices. When a piece of technology suddenly sells extremely well, by default it may become a “standard”. An example of a controversial “standard” is the file formats used by Microsoft. These are used internationally, by possibly 85% of computer users, but because the details of these standards are not fully disclosed other providers of computer programs can never quite create complete compatibility between their programs and the Microsoft equivalents. These are “closed standards” – they exist, but not all the details of those standards are known.

“Open standards” are routinely set by organisations like the IEEE (www.ieee.org), a non-profit organisation that works with global groups of people to establish standards. The IEEE claims to have 370 000 members in over 160 countries, and when it sets a standard most organisations will conform to it, thereby facilitating the interface between different pieces of technology. An open standard is publicly available for others to use. In the programming world “open” is often taken to mean “royalty-free” and in some countries, open standards are legislated to be free of charge to users. Open formats relate more to the specifications for storing digital data such as word processor documents and databases.

The qualifications framework approach to standards

Since the establishment of the first generation of national qualifications frameworks in Australia, New Zealand, the United Kingdom and South Africa in the 1990s, qualifications frameworks have become a global phenomenon. By 2008 more than 60 countries across the world, including most European Union member states and Sub-Saharan African countries, are at different stages of developing national qualifications frameworks, with at least three regions having made progress towards regional qualifications frameworks (Coles 2006). While it is common knowledge that the development of qualifications frameworks has not remained uncontested (Donn & Davies 2003, Young 2005, Keevy 2005, Allais 2007), it is becoming increasingly evident that qualifications frameworks do offer significant advantages in terms of recognition and comparability of qualifications, if developed and implemented appropriately.

The transnational qualifications framework for the VUSSC will in some ways be similar to a regional qualifications framework, such as is being developed in Europe) , Southern Africa (SADC 2005) and the Caribbean . Whereas a regional framework tries to address the interests of a particular region’s culture, languages and issues, the transnational qualifications framework attempts to address the idiosyncrasies of small and island states. The development of the transnational qualifications framework will include collaboration with regional authorities and help to stimulate networking between regions, in support of the countries with the smallest economies and populations on earth.

There is no doubt that qualifications framework development, be it national, regional or transnational, can draw on 21st century thinking and technology to serve the emerging needs of stakeholders. The development of information technologies since the conception and implementation of the first frameworks is nothing short of revolutionary. Similarly, the way that work is conceived and organised and carried out, the skill sets required for working in today’s world, and the implications of all this for education provision and management, represent a complete paradigm shift.

A particular aspect of qualifications frameworks that is being challenged through the development of new technologies is the conventional notion of a standard. It is argued that the conventional approach is biased towards the development of relatively inflexible and “closed” standards that remain constant over time, with limited opportunity to respond to the continually changing external environment, even less so to the continuous striving for improvement required from the internal environment. While this argument for open and dynamic standards within the qualifications framework environment makes sense at face value, it is not without flaws. Closer interrogation of the concept of a standard, as well as the extent to which expert practitioners are needed during the developmental process, suggests that the way in which standards are interpreted from the world of “wikis” conflates a number of important conceptual considerations.

In the world of qualifications frameworks, standards specify, often through outcome statements, the skills, knowledge, values and attributes which learners should achieve in order to obtain a particular qualification. Standards are used as a basis for the development of qualifications and curricula (top-down), and in some cases, are developed from existing curriculum and qualifications (bottom-up). Standards are defined in a variety of ways across countries and regions, which use terms such as qualification standards, competency standards, vocational standards and unit standards. High value is placed on stakeholder involvement during the development of standards, as in South Africa, where emphasis is placed on the descriptions of learning achievements “agreed by all major stakeholders in the particular area of learning” (South Africa 1998). In addition, standards in the world of qualifications frameworks attempt to place the specification of what learners should be able to achieve in the public domain, as emphasised in the Australian case (Gunning 2004).

The use of standards in the world of qualifications frameworks is by no means uncontested. Outcomes statements are seen as too limiting, often leading to over-specification when attempting to describe tacit skills, particularly when considered from a behaviourist paradigm. The top-down approach to standards development (the dominant model used in the first generation of qualifications frameworks) is critiqued for being removed from the context wherein learning takes place. Stakeholder-driven models are criticised for neglecting the role of expert practitioners located in specific communities of practice. The production line approach (in the Fordist tradition) employed for the development of standards is viewed as outdated and inflexible. And so the list continues.

The debates on standards highlight a range of key considerations for second- and third-generation qualifications frameworks that, at least at face value, can be informed by the world of open and dynamic standards. For example, it makes sense to consider how conventional standards can be developed in a more dynamic and flexible manner. In another example, the involvement of non-experts can be considered to strengthen the standards development process. In both examples, a closer interrogation shows that similar considerations have already been explored (even during the development of the first generation of qualifications frameworks), but with varying degrees of success. Considering the first example, it is immediately apparent that it is inaccurate to consider that the conventional approach to standards development is closed. In most cases significant care is taken to involve various stakeholders extensively during the process, including providing opportunities for adjustments at specific intervals. Standards are registered on qualifications frameworks for fixed periods, mostly within the boundaries of review dates. In some cases review periods are fixed for all qualifications (e.g. South Africa and Ireland), while in others a more flexible and differentiated approach is followed that allows for more frequent reviews of standards, such as in information technology (e.g. Scotland). In the case of the second example, the involvement of stakeholders including non-experts, such as representatives from industry, unions, professional associations and government, in standards development has been foregrounded in most qualifications frameworks. Indeed, such involvement has been extensively criticised (for example in South Africa), resulting in a return to more expert-driven standards development models.

Considering the colliding worlds: concluding comments

Returning to the transnational qualifications framework case study that prompted this debate on open and dynamic standards, it is our view that the current expert-driven approach to qualifications development, even though it is contested in some areas, is followed during the initial stages of the development of the a transnational qualifications framework. It is our view that developing VUSSC qualifications through open and dynamic standards, though this presents some useful alternatives, is not viable at present. Even so, open and dynamic standards present some important considerations for the development of the VUSSC qualifications within the context of the proposed transnational qualifications framework. In particular it is evident that clear, non-negotiable standards are warranted at least for certain fixed periods (e.g. six months, a year or even a few years, depending on the specific field). Nevertheless the ability to respond and adapt to a continually changing environment is equally important, and requires periodic reviews of qualifications. Furthermore, standards-setting bodies cannot be exclusively responsible for qualifications development and need to allow for contributions from non-experts, though not to the extent that non-experts are able to unduly influence the process.

There is no doubt that “wikis” and open and dynamic standards have much to offer the world of standards development and qualifications frameworks. These offerings must, however, be considered with caution. Closer interrogation of both “approaches” to standards development presented in this paper suggests that the two worlds are not that far apart. Qualifications frameworks, as a fairly recent phenomenon, and the world of open and dynamic standards, as an even more recent phenomenon, face similar challenges when confronting the more traditional notions of qualifications development. It is however evident that although the qualifications framework discourse has embraced and interacted with such criticisms, the more fluid discourse of open and dynamic standards has yet to do so.

References

Allais, S. 2007. Education service delivery: the disastrous case of outcomes-based qualifications frameworks. Progress in Development Studies 2007(7): 65.

Commonwealth of Learning and South African Qualifications Authority (COL & SAQA). 2008. Transnational qualifications framework for the Virtual University of Small States of the Commonwealth. Forthcoming.

Coles, M. 2006. Review of international and national developments in the use of qualifications frameworks. Report for the European Training Foundation, April 2006. Discussion paper for the European Training Foundation, March 2006.

Domingue, J, Julie, H, Keevy, J & Mphuthing, D. 2008. Occupational qualifications in the Seychelles and South Africa: a critical reflection on progress to date. Paper presented at the Q Africa 2007 conference, Gallagher Estate, Midrand, South Africa, 22-23 November 2007.

Donn, G & Davies, T. 2003. Promises and problems for Commonwealth qualifications frameworks. Report of a Commonwealth seminar held in Wellington, New Zealand, February 2003.

Gunning, D. 2005. The role of qualifications in improving pathways. In: The Policy Context for Education & Training 29 October 2004. Ascot House: Melbourne.

Keevy, JA. 2005. A Foucauldian critique of the development and implementation of the South African NQF. PhD thesis. University of South Africa.

Ministry of Education of the Federal Democratic of Republic of Ethiopia. 2008. Conceptual framework for the development of the Ethiopian National Qualifications Framework. Unpublished consultative document.

SADC Technical Committee on Certification and Accreditation (TCCA). 2005. Towards a Southern African Development Community Qualifications Framework. Concept paper and guidelines. Unpublished concept document presented to the SADC Council of Ministers, June 2005.

SADC Technical Committee on Certification and Accreditation (TCCA). (2007). Benchmarking quality assurance systems in SADC Member States. Forthcoming.

SAQA & Commonwealth Secretariat. 2006. The recognition of teacher qualifications and professional registration status across Commonwealth member states. Pretoria: SAQA.

Young, MFD. 2005. National Qualifications Frameworks: their feasibility for effective implementation in developing countries. Report prepared for the International Labour Office (ILO). Unpublished.

Work in progress, expect frequent changes. Help and feedback is welcome. See discussion page.