User:Prawstho/WikiQuality

Assessing the quality of open education resource based wikis

Author(s): Peter Rawsthorne, it would be nice to have others add to, edit this page...

Work in progress, expect frequent changes. Help and feedback is welcome. See discussion page.

Abstract

The internet has opened up the ability for educational materials to be developed in a distributed and internationally collaborative environment. These capabilities have manifested themselves in collaborative “wiki” websites. These sites contain the contributions of authors from around the globe adding to, updating and editing each others work. Recently this collaborative approach has found itself being used for the creation and deployment of Open Educational Resources (OER). This approach is evidenced by the development of wiki based OER’s, such as; Wikiversity, Curriki, and WikiEducator. These educational wikis are quickly growing in size and popularity and for these free educational materials to deepen their success and sustainability their quality needs to be continually scrutinized. It is through scrutinization that these OER based wikis will; have increased accuracy, implement appropriate pedagogical approaches, have longer-term universal usefulness and be easily customized to different languages, cultures and contexts. Currently, there is a lack of research exploring the issues of educational and information quality within the context of the OER based wikis. Though, there are many initiatives and approaches investigating the quality and self-organization of collaboratively developed online content. Many of these can be brought together and utilized to evaluate the quality of wiki based OER’s. This paper provides a quality assessment framework focused upon OER based wikis. This paper has four main sections; first, it begins with a review of OER and how it is being implemented in wikis. The second section is twofold; it evaluates the approaches to assessing educational quality and the assessment frameworks available for education and information quality. And it investigates the growing assortment of innovative approaches for determining the quality of wiki based content. Third, it takes from these approaches, frameworks and innovations to create an assessment framework targeted toward wiki based OER’s. Fourth, the paper offers some next steps and conclusions of how best to implement a framework for assessing the quality of wiki based OER.

Background

The internet has initiated many social and global changes. One of the more recent and most successful is the open source movement. Wikipedia (2007) describes open source as a set of principles and practices that promote access to the design and production of goods and knowledge. The definition continues to describe how the open source model of operation can be extended to open source culture in decision making, which allows concurrent input of different agendas, approaches and priorities, in contrast with more centralized models. This move away from centralization is important as Neus (2001) points out that the accelerating need for both time-critical and high-quality information, the traditional means are often no longer sufficient. The combination of openness in both software and standards has created an information infrastructure that is supporting an unprecedented volume of information creation. It is the internet tools of blogs and wikis that have introduced so many people to the activity of collaborative content creation. These tools and approaches are finding their place in educational institutions where, higher education is often keen to embrace openness, including new tools such as blogs and wikis for students and staff. (Kelly, Wilson & Metcalfe; 2007) As the collaborative creation of information continues it will become important to have methods to ensure the quality of all this information. Due to the velocity of information creation and the collaborative nature of these new internet tools it would make sense to have information quality practices built into the tools and approaches. As Neus (2001) successfully argues ‘It is this scarce resource, the human attention, which we must learn to better manage and direct toward the high-quality, relevant information in the exponentially growing haystack of low-quality information in collaborative environments.’ Quality is also tied to reputation and therefore becomes critically important for higher education institutions generally, but especially for institutions involved in open and distance learning (Inglis, 2005). With the internet has also come cross-border education. Students living in one country can complete a university program in another country without having to leave their home. Cross-border education has increasingly brought the issue of educational quality to the international standards bodies where in 2005, UNESCO published “guidelines for quality provision in cross-border higher education”.

Open Educational Resources

In general, Open Educational Resources (OER) are educational material that are open and can be copied, reused, altered and derived without cost. Keats (2003) brings the importance of OER into an international development context by stating that;

the cost of content, the under-resourcing of universities and the scattered nature of expertise in Africa, the collaborative development of open content seems like a useful way to get high-quality, locally-relevant content for using to enhance teaching-and-learning.

To ensure these materials are used will vary depending upon the copyright licence applied to the OER. The idea of “open” implies there is no cost to use the materials, and the type of copyright licence specifies how the materials can be used. There is considerable dialog occurring within the OER movement regarding the definition of open and related licensing approaches, as Downes (2007) states, the lack of a shared understanding allows the definition of 'open' to be stretched in a manner that would discomfort many advocates. Due to this flexibility, and sometimes controversy, it becomes important to understand the subtleties when choosing a licensing method for OER. There are two primary licensing methods to release open materials, these are Creative Commons (CC) and GNU Free Documentation Licence (GFDL). The creative commons license allows for the individual or group releasing the materials to choose their licensing approach, the Creative Commons has four options; 1) Attribution. You let others copy, distribute, display, and perform your copyrighted work - and derivative works based upon it - but only if they give credit the way you request. 2) Noncommercial. You let others copy, distribute, display, and perform your work — and derivative works based upon it — but for noncommercial purposes only. 3) No Derivative Works. You let others copy, distribute, display, and perform only verbatim copies of your work, not derivative works based upon it. 4) Share Alike. You allow others to distribute derivative works only under a license identical to the license that governs your work. The GNU Free Documentation Licence or GFDL, like the Creative Commons license has been created to promote the free sharing of materials and to allow the creators to specify how the materials can be used. The definition provided on wikipedia regarding the GFDL (2007) provides information on how this licensing approach should be used.

The GNU Free Documentation License (GNU FDL or simply GFDL) is a copyleft license for free documentation, designed by the Free Software Foundation (FSF) for the GNU project. It is the counterpart to the GNU GPL that gives readers the same rights to copy, redistribute and modify a work and requires all copies and derivatives to be available under the same license. Copies may also be sold commercially, but if produced in larger quantities (greater than 100) then the original document or source code must be made available to the work's recipient. The license was designed for manuals, textbooks, other reference and instructional materials, and documentation which often accompanies GPL software. However, it can be used for any text-based work, regardless of subject matter. (“GFDL”, 2007)

Having clarity with choosing copyright has greatly assisted in increasing the availability and importance of open educational resources (OER) and therefore we have seen a steady increase in the use of OER (Downes, 2006; Hylén, 2006; Wiley, 2006). Many educational and development leaders see the use of OER at a global scale as very important in assisting in decreasing the global divide in education access. (WikiEducator, 2007)

The Wiki

The wiki is a software application that allows multiple authors to add and update content. In general, the content is made available on the internet and the authorship is an activity of mass collaboration from many geographic locations and time zones. The wiki has a syntax and infrastructure that allows authors to easily add and update content while also participating in a constructive dialogue about the multi-authored content. One of the strengths of the wiki is how well suited it is for constructivist learning approaches. Parker and Chao (2007) speak to reflective learning, as one of the critical features of constructivism; this refers to structured approaches that enable students to reflect upon their learning and to understand their own learning processes. Wikis are playing an increasing role in students’ reflective learning, and enrich students’ experience. The wikis collaborative foundation also fits well within socio-constructivist and community of practice approaches where groups of learners use the wiki to build knowledge together (Boulos, Maramba, & Wheeler, 2006; Lamb, 2004). There is no doubt the wiki is growing in popularity as a learning tool within academia domestically, internationally and inter-country (UNESCO, 2005). Using the open and collaborative wiki has been identified as an approach to meet the OER needs of the developing world. If we are to have a free and universally available curriculum by 2015 (Commonwealth of Learning, 2006) we need new approaches to achieve this goal, the educational wiki is showing considerable promise as evidenced by the growing amount of content within WikiEducator and Wikiversity. Downes (2006) is a strong and vocal advocate for OER and believes that OER use could be improved most effectively through a shift from a ‘provider/user’ paradigm to a community model of collaborative development. If one is to doubt the ability for a wiki to develop quality content they would have to look not further than Wikipedia which has two employees and well over a million articles in multiple languages. As Wiley (2005) states; we need to learn this lesson if open education is really going to reach out and bless the lives of people.

Wiki authorship, social-constructivism and self-organization

During the introduction of Parker and Chao's (2007) paper, 'Wiki as a teaching tool' they reference many authors including Jonassen and Wilson in stressing the social and collaborative aspects of constructivist activities within wiki based social software. The connection and effectiveness between social software, wikis and constructivism has been well documented and an in-depth discussion of this topic is beyond the scope of this paper. It is important to recognize that the wiki is considered an effective and proven socio-constructivist learning environment. As we get deeper into understanding the wiki and for the purpose of this paper the attributes of the wiki in relation to educational quality there are three important themes, these include; mass authorship, socio-constructivism and self organization. The importance of mass authorship or collaboration is in its increased adoption and in how the wiki approach encourages a focus on content rather than the technology. When it comes to adoption, Tapscott and Williams (2006) go as far to say “This new participation has reached a tipping point where new forms of mass collaboration are changing how goods and services are invented, produced, marketed, and distributed on a global basis”. The transparency of the wiki technology encourages content creation and its ability to allow the user to concentrate more on the learning task by 'seeing through' the technology with which they are interacting. (Parker & Chao, 2007). The socio-constructivism becomes important for it is the community that both creates the content and learns as it builds, edits, reverts and improves the wiki articles. Again, Parker and Chao (2007) provide insight into how the wiki tools allow the user community to monitor the constantly changing state of the wiki and discuss the issues that emerge. The collaborative learning becomes even more powerful when it takes place in the context of a community of practice. A community of practice consists of people engaged in collective learning in a shared domain. One of the interesting and unexpected attributes of the wiki is in its self-organization. Many believed that anarchy (Stvilia, Twidale, Gasser & Smith; 2005) would break out if you allowed an internet information infrastructure (the wiki) to be open to anonymous editing. What has happened is quite the opposite; it has become known as the artificial information economy “which discourages low-quality or offending input, because it is much “cheaper” for person B to undo the low-quality change that person A caused, than it is for person A to cause it. This process weeds out low-quality information in an evolutionary paradigm (Neus, 2001). The self-organizing qualities of the wiki encourage community introspection: that is, it is strongly designed so that members watch each other, talk about each other’s contributions, and directly address the fact that they must reach consensus (Viégas, Wattenberg, & Kushal; 2004).

Quality Assessment

In this paper we focus on the issues of quality within education. Quality has been having a steady growth of attention due to the increase in cross-border education and the use of OER. As Stella (2006) points out that “During discussions on quality assurance related issues, especially when it concerns ‘death of distance’ and ‘diminishing geographical borders’, very often cooperation among quality assurance agencies comes up as a suggestion to work towards a solution”. When working toward quality in an international and collaborative environment it is important to recognize the differences inherent in a cross-border and cross cultural endeavour, for it is these differences that define the context and assumptions behind the quality assessment. Tam (2001) writes extensively about how quality is a ‘relative concept’ and “to understand quality it is necessary to recognise that it has contradictory meanings that can lead to different assessment methods and thus different practical outcomes”. This supports the idea that for quality education to grow within an international context there needs to be a supporting infrastructure to encourage the quality dialogue.

Within this paper three different views of quality will be explored; educational quality, informational quality and wiki quality. It is the intersection of these three views that offers the opportunity to identify a new framework focused upon the quality of an educational wiki. Where each view has a different aspect of quality, it is also important to address certain attributes of a view that is not within the intersection. An example would be the requirements of the learner would not be within assessing information quality, yet it would be an attribute of education quality. The framework for assessing quality within an educational wiki needs to recognize this (and other) attribute(s) to be viable.

Educational Quality

There is no shortage of critical information regarding educational quality. There are journals dedicated to the subject of quality in higher education. The subject of this paper is the activities focused around the quality of international, cross-border and open education. The focus is also on OER, as Johnstone (2004) reiterates statements made by UNESCO; that OER was defined to include the resources to assure the quality of education and educational practices. It therefore becomes important to describe; what aspects of quality apply to international education, in particular, OER? And what is important and needs to be considered when developing a framework for assessing the quality of OER based wikis?

As previously mentioned it is difficult to describe quality as it is influenced by many factors, and once a localized description of quality is agreed upon it is difficult to define methods to measure or assess quality. Both Hylén (2006) and Barbera (2004) explore the issues of quality in regards to education and both their discussion focuses upon the dissonance amongst learning, educational resources, the identification of quality, institutional providers and achievement. Hylén and Barbera also agree that quality should occur in a constructivist way where the learner independently decides what quality is and how it should be determined within their own context. This more independent approach leads to a framework for quality that is more subscriptive than prescriptive. It isn’t how you DO quality, it is that you are DOING quality, and the how is left to the user, institution or organization. This challenge is one of the main reasons why international guidelines have been published by UNESCO (2005). These guidelines provide a comprehensive set of suggested practices to increase the quality of education whether local, national or international. When it comes to creating quality within OER it becomes a subscriptive process defined by the users in collaboration with the institutionalized provider. As Downes (2006) states, “OER use could be improved most effectively through a shift from a ‘provider/user’ paradigm to a community model of collaborative development”.

Approaches

Measuring and determining quality requires an approach or methodology. As has been previously discussed, quality is done within a context and the context should determine the approach. Inglis (2005) has identified three terms (or approaches) that commonly appear in relation to quality in education, these are; benchmarking, quality assurance, and quality improvement. The approaches described within these three terms lend themselves very well to assessing the quality of OER.

A benchmark would involve comparing educational resources to the best that can be found within the relevant context. It would be important to ensure the comparison would be between like educational resources while being mindful of context. An example of a quality benchmark would be comparing the educational resources for sustainable logging practices from British Columbia, Canada against the best educational resources of sustainable logging practices from Sweden. This would not provide a fair comparison for the context and practices would be different as defined by the culture and local practices. It would be best to benchmark the educational resources within a localized context. Benchmarking the quality of OER requires that the comparison stays within context.

Quality assurance is process oriented in that the quality of an educational resource meets some predetermined standard. Quality assurance makes no assumptions about the quality of some other educational resource. In practice, however, quality assurance standards would be expected to reflect norms for the relevant context. In following the sustainable logging practices example the educational resource in British Columbia would be compared to the standards held within the local logging industry, the educational standards set by the local and national education standards and by the standards set by the educational institution. The quality assurance process would be in meeting these standards. As standards changed within the industry and educational institutions needs so would the quality assurance process would also need to change.

Quality Improvement is concerned with raising the quality of the educational resources. The type of comparison that is made when engaged in quality improvement is between the current standard of the educational resources and the standard being aimed for. Quality improvement is concerned with comparing the quality of what is about to be produced with the quality of what has been produced in the past. Again following the sustainable logging practices example the current educational resources in British Columbia would be compared to the emerging logging, educational and institutional standards. The changes to the educational resources would be for improvement.

When choosing a quality assessment approach context is very important, particularly when considering the cross-border issues within OER. When using any of these quality assessment approaches context should be considered early in the design of the assessment. All three approaches lend themselves to be used within the development of quality practices for OER based wikis. Benchmarking is well suited to assessing educational resources against what is considered to be best. Quality assurance is well suited to setting a baseline standard for educational resources and having them meet the baseline of what is considered viable learning materials. And Quality improvement is well suited to improve the overall quality of the educational resources through time as changes to standards and educational approaches change.

Frameworks

Whether you are seeking a benchmark, quality assurance or quality improvement the quality assessment framework provides a structure in which to affect quality. The purpose of the framework is to provide the set of guidelines, practices and activities when followed will determine or improve the level of quality. Barbera (2004), Houston (2007), Inglis (2005), Gosling & D'Andrea (2001) all write about the importance of creating frameworks that are localized. Houston (2007) is most concise when he states; It is likely to be far more fruitful to explore the development of locally appropriate approaches to improving quality in and of education. Localized quality assurance is the preferred approach and upon review of the frameworks proposed by Barbera (2004), Inglis (2005) and UNESCO (2005) all support localization. Which framework you choose will depend upon the context the framework will be used and the views of quality held by the stakeholders. The challenge will not be in finding a framework; the challenge will be in choosing and adapting a framework to work within the localized context.

Information Quality

The quality of the information we use in decision making is critical to make well informed and high-quality decisions. Information Quality (IQ) is the term used to describe the quality of the content of information systems (“Information Quality”, 2007). As with other discussion around quality, context is important in defining quality. For IQ, the context is brought into the equation through the use of the information’s ‘fitness for use’ (Bovee, 2003; Neus, 2001) or ‘fitness for purpose’ (Harvey, 2002; Houston, 2007). The concept of ‘fitness’ is in how the information is useful and is it to a level of accuracy that is appropriate for the context or domain it is being used. Within a wiki this becomes important for the “fitness” or quality is determined by the community who uses, edits, discusses and agrees upon the information. Given the relative immaturity in using academic rigour in researching and measuring the information quality of wikis there are few people to refer in the space of information quality and wikis. The group of Stvilia, Gasser, Twidale and Smith working out of the Graduate School of Library and Information Science at the University of Illinois have made the most progress in this area. Their research in this area is setting the tone for assessing the information quality of wikis. The (Stvilia, Twidale, Gasser & Smith, 2005) approach is best described in their writings;

We examine the Information Quality aspects of Wikipedia. By a study of the discussion pages and other process-oriented pages within the Wikipedia project, it is possible to determine the information quality dimensions that participants in the editing process care about, how they talk about them, what tradeoffs they make between these dimensions and how the quality assessment and improvement process operates. This analysis helps in understanding how high quality is maintained in a project where anyone may participate with no prior vetting.

When it comes to assessing the quality of the information within a wiki it becomes the participants involved in the editing of the wiki page that determine the fitness for use of the information. This participatory approach also fits well within the definition of educational quality for it is the participants that will localize the content to be within desired level of quality.

Approaches

Given the importance of the community and its participants (Smith, 1999); the richness of information found within a wiki's pages, edit histories and discussion pages; and the viability of readability and dialogue analysis it becomes the combination of these that create the approach to assessing the information quality of wikis. Stvilia, Gasser, Twidale & Smith (2005) create an approach (model) to assessing the IQ of wikis by considering all these previously mentioned attributes. They go as far to say “One cannot manage IQ without first being able to measure it meaningfully and establishing a causal connection between the source of IQ change, the IQ problem types, the types of activities affected, and their implications”. It is from here that they developed a framework to measure the IQ of wikis, in particular Wikipedia.

Frameworks

The approach suggested in this paper is to leverage the research and IQ assessment model created by Stvilia, Gasser, Twidale and Smith (2005) and create a framework for evaluating the information quality of OER based wikis. Their framework has seven IQ metrics that can be altered (or reorganized to allow for greater detail) as the quality context changes. Their assessment model consists of seven metrics;

1. Authority/Reputation - this metric determines the 'maturity' of the information source. Number of authors, edits, reverts, how connected is the article. This metric is closely aligned with what is measured with the history flow spoken of later.
2. Completeness - This metric evaluates the length, and interconnectedness of the article.
3. Complexity - evaluates the complexity of the article based upon a readability score, such as the Flesch readability score. (“Flesch Reading Ease,” 2007)
4. Informativeness - conciseness (or lack of noise), the diversity of content within the subject and use of images and supporting rich media.
5. Consistency - the frequency and number of administrator edits in relation to other authors and the age of the article.
6. Currency - the age of the article
7. Volatility - the median revert time or time between edits.

Wiki Quality

This paper explores the quality of OER based wikis within the international context. As previously stated it becomes important to recognize context (language, culture, etc.) and the different meanings of quality related to context or ‘fitness for use’. The collaborative dialogue which occurs regarding the quality of an article (or learning module within OER) becomes an important attribute of measuring the quality. This pattern of dialogue directed toward the quality of an item follows the pattern of discussion of a wiki article. This idea of measuring a wiki articles quality based upon discussion pages is confirmed in Hylén’s (2006) paper where it is shown how open and decentralized OER (the wiki) quality can be determined through user comments and ratings. The effectiveness of utilizing the users of the wiki as the quality control community follows from what has become known as Linus’ Law (Linus Torvalds, originator of the Linux operating system) which states, ‘Given enough eyeballs, all bugs are shallow’. To put this into the wiki based content perspective it would become ‘Given enough contributors, all information becomes accurate’. The ability for contributors to subscribe to the articles they have contributed is a built in feature of most wiki environments. This allows the contributors to be informed whenever a subscribed to article changes. What comes from this is a continuous inspection of an article and therefore increases its information economy. It could be considered that every wiki article exists in a state of continuous quality improvement.

Approaches

Throughout this paper approaches to quality have been discussed. As the focus has moved toward the information quality of wikis, four main themes have emerged. These include; first, having many people viewing the articles, improves the quality of articles through time. Second, the edit and revert cycles supported by strong dispute resolution (“Dispute Resolution”, 2007) increases article quality. Third, the article edit cycles serve as a process of continuous quality improvement. And fourth, contributors self-organize when it comes to the quality and accuracy of articles. An interesting innovation in visualizing the edit, revert and dispute cycles comes from a software tool known as history flow visualization. History flow is a new exploratory data analysis tool developed by IBM and Viégas, Wattenberg, and Kushal (2004). The tool is effective in revealing patterns within wiki context. The visualization focuses on the relevance of authorship, the value of community surveillance in ameliorating antisocial behavior, and how authors with competing perspectives negotiate their differences. The history flow tool provides a data analysis infrastructure that addresses concerns of failure to accurately measure, map and analyze the quality of virtual communities and social networks brought forward by Barbera (2004) and Neus (2001).

The strength of the wiki based virtual community comes in the self-organization and how this improves the quality of the context. Within a learning environment Hylén (2006) recognizes that quality is not an inherent part of a learning resource, but rather a contextual phenomenon. It is only in the specific learning situation that it can be decided whether a resource is useful or not, and therefore it is the user who should be the judge. When you consider that it is the same group of collaborators creating and judging the content it then becomes possible for a loosely knit network of skilled amateurs to produce comparable or better quality information in a collaborative paradigm than traditional solitary authors, institutions or publishers are able to create (Neus, 2001)

In the end, the best approach to the continuous quality improvement of wiki and OER based content is to have groups of dedicated and collaborative authors using the tools, processes and self organization to provide an “organic” approach to quality improvement.

Frameworks

Two authors that have had significant influence over the approach discussed in this paper have been Barbera and Neus. Barbera (2004) provides a framework with a perspective aligned toward educational quality where Neus (2001) provides a framework with a perspective aligned toward information quality. The synergy within the educational and information quality within these two frameworks is compelling;

“Six dimensions for measuring quality in the virtual environment, these include; Analysis and evaluation of the educational situation, Processes followed by participants, Analysis of the agents and their roles, Interaction within the virtual environment, Instruments used for interaction and the Knowledge built from the education process” (Barbera, 2004).

The challenge then becomes one of creating such an information economy that produces high quality information. Some important cornerstones for this to work are the following five factors:

1. Accountability for contributions as a basis for reputation
2. A thematic focus and “culture” for high quality contributions
3. A sense of trust and identity through personal profile pages
4. A common memory or knowledge repository which is developed in collaboration
5. Membership criteria to keep the level of discourse high and on topic

(Neus, 2001)

The synergy gained from these two frameworks creates the following five themes to be considered when creating a framework for assessing the quality of open educational resource based wikis;

1. Shared processes within collaborative virtual environments
2. Membership and well defined roles
3. Knowledge of themes, culture and situation
4. Importance of building collaborative knowledge and learning repositories
5. Aspirations toward evaluated and high quality content

Quality Assessment of OER based Wikis

The quality assessment of OER based wikis drawns upon the assessment methods for educational quality, information quality and wiki quality. It is the union of these three that provide the framework for the quality assessment for OER based wikis. The Wiki based OER quality framework being proposed by this paper takes what is important from all three of these reference methodologies. The important factors from educational quality include; utilizing the self-directed and collaborative approaches found in constructivism and socio-constructivism (Barbera, 2004; Hylén, 2006; Lamb, 2004), the narrative and reflective writing as a constructivist activity (Boulos, Maramba, & Wheeler, 2006; Parker & Chao, 2007; Yukawa, 2005), the distinction of benchmarking, quality assurance and quality improvement (Inglis, 2005) and localizing the quality assessment method to ‘fitness for purpose’ for cross-border education (Barbera, 2004; Bovee, 2003; Harvey, 2002; Houston, 2007; Inglis, 2005; Gosling & D'Andrea, 2001; Neus, 2001; UNESCO, 2005). The important factors from information quality include; ‘fitness’ for the localized context or domain (Bovee, 2003; Harvey, 2002; Houston, 2007; Neus, 2001; UNESCO, 2005), participant involvement and the wiki structures of edit/revert, history and discussion (Stvilia, Gasser, Twidale & Smith, 2005). The important factors from wiki quality include; quality assessment based on history and discussion pages (Hylén, 2006; Stvilia, Gasser, Twidale & Smith, 2005), Linus’ Law of ‘given enough eyeballs, all bugs are shallow’, The mature dispute resolution process (2007) defined by wikipedia, and the history flow visualizations provided by Viégas, Wattenberg, and Kushal (2004). What is particularly intriguing about the union of these three quality approaches is that where one leaves off the other starts up. This is apparent through the connection of the participant data required by history flow is the same data that is created by the virtual communities and social networks present within the socio-constructivist learning environments.

Approaches

When defining an approach to defining quality within an educational environment both Barbera (2004) and Inglis (2005) believe that the framework for measuring quality must be contextualized and view the whole system as an integrated reality. The approach to assessing the quality of OER based wikis will follow this view and include the holistic reality defined by the institution creating the OER based wiki and the collaborators who create and define the content and its quality. To further entrench our approach in educational quality practices we will draw upon two important approaches when defining the OER based wikis quality. This will be to apply quality assurance practices to the educational institution (i.e. Wikiversity and WikiEducator) and quality improvement practices to the wiki content. This alignment is due to the institution being better suited for quality assurance practices and the wiki content being better aligned to quality improvement practices.

Framework

The framework to assess the quality of OER based wikis will be threefold; first and foremost it will focus on the depth of learning and knowledge acquisition by the student or participant; second, is the education institution continuing toward increasing its quality assurance; and third is the content of the wiki always moving toward quality improvement. To facilitate the success of using this framework the education institution (virtual or otherwise) should be considered the institution providing the infrastructure and services to host and create the OER based wiki(s). There level of involvement in creating the OER will differ from project to project, context to context and within the different communities of practice. The idea of building traffic is the responsibility of the institution and the participants. The wiki based quality assessment tools available should be used to encourage participants to review, create, edit, evolve and reuse wiki content. The quality assessment tools include the framework proposed by Stvilia, Gasser, Twidale and Smith (2005) and the history flow tool developed by IBM and Viégas, Wattenberg, and Kushal (2004).

Discussion

The union of educational, information and wiki quality contains many approaches shared by all three of these quality assessment frameworks. The frequency in which attributes of one framework are mirrored in another shows the appropriateness of applying features of one framework within the whole. It therefore becomes the responsibility of the institution creating the OER based wiki to initiate the tone and encourage the dialogue regarding the quality of the wikis content. It is also important that the ongoing discussion includes context, which includes language, culture, etc. when defining and redefining the OER based wiki. For in the end, the OER based wiki is meant for the participants to increase the quality of their knowledge in a participatory and socio-constructivist environment.

Future Work

The approaches and frameworks described within this paper are ideas and observations that need to be proven out in practice. The future work to be born out of this paper is to apply what has been discussed with the participant communities forming in the OER based wikis of WikiEducator and Wikiversity. It is from this engagement that the quality of an OER based wiki assessment tool can be assessed and improved.

Conclusions

Self-organization has become recognized as a powerful and viable approach to affecting the quality of open resources. Wikis have become one of the internet environments most influenced by self-organization. Using the practices found within self-organization to improve the quality of OER based wikis is potentially the most viable approach to building a complete OER based curriculum by 2015.

References

Barbera, E. (2004). Quality in virtual educational environments. British Journal of Educational Technology 35(1), 13–20.

Boulos, M.N.K., Maramba, I., & Wheeler, S. (2006). Wikis, blogs and podcasts: A new generation of Webbased tools for virtual collaborative clinical practice and education, BMC Medical Education, 6(41). Retrieved on July 24, 2007 from http://www.biomedcentral.com/content/pdf/1472-6920-6-41.pdf

Bovee, M., Srivastava, R. and Mak, B. (2003). A Conceptual Framework and Belief-Function Approach to Assessing Overall Information Quality. International Journal Of Intelligent Systems. 18, 51–74

Commonwealth of Learning. (2006). Education for All. Retrieved on 3 July, 2007 from http://www.col.org/colweb/site/pid/4006

Creative Commons. (n.d.). Choosing a License. Retrieved June 14, 2007, from http://creativecommons.org/about/licenses/

Curriki. (n.d.). Global Education and Learning Community. Retrieved 13 March 2007 from http://www.curriki.org/xwiki/bin/view/Main/

Dispute Resolution. (2007, July 27). In Wikipedia, The Free Encyclopedia. Retrieved July 27, 2007, http://en.wikipedia.org/wiki/Wikipedia:Dispute_resolution

Downes, S. (2006). Models for Sustainable Open Educational Resources. Interdisciplinary Journal of Knowledge and Learning Objects 2006, 3, 29-44. Retrieved on May 23, 2007 from http://www.ijklo.org/Volume3/IJKLOv3p029-044Downes.pdf

Flesch Reading Ease. (2007, July 17). In Wikipedia, The Free Encyclopedia. Retrieved July 21, 2007, from http://en.wikipedia.org/wiki/Flesch-Kincaid_Readability_Test#Flesch_Reading_Ease

GNU Free Documentation License. (2007, June 28). In Wikipedia, The Free Encyclopedia. Retrieved July 2, 2007, from http://en.wikipedia.org/wiki/GFDL

Gosling, D. and D'Andrea, V. (2001). Quality Development: a new concept for higher education. Quality in Higher Education, 7(1) 7-17.

Harvey, L. (2002). Evaluation for What?. Teaching in Higher Education, 7(3) 245 - 263

Hylén, J. (2006). Open Educational Resources: Opportunities and Challenges. Retrieved on May 23, 2007 from http://www.oecd.org/dataoecd/5/47/37351085.pdf

Houston, D. (2007). TQM and Higher Education: A Critical Systems Perspective on Fitness for Purpose, Quality in Higher Education, 13(1) 3-17.

Information Quality. (2007, July 5). In Wikipedia, The Free Encyclopedia. Retrieved July 13, 2007, from http://en.wikipedia.org/wiki/Information_quality

Inglis, A. (2005). Quality Improvement, Quality Assurance, and Benchmarking: Comparing two frameworks for managing quality processes in open and distance learning. International Review of Research in Open and Distance Learning, 6(1). Retrieved on May 24, 2007 from http://www.irrodl.org/index.php/irrodl/article/view/221/867

Johnstone, Sally M. Open Educational Resources Serve the World. 2005. Educause Review. http://www.educause.edu/apps/eq/eqm05/eqm0533.asp

Keats, D. (2003). Collaborative development of open content: A process model to unlock the potential for African universities. First Monday, 8(2). http://firstmonday.org/issues/issue8_2/keats/index.html

Kelly, B., Wilson, S. & Metcalfe, R. (2007). Openness in Higher Education: Open Source, Open Standards, Open Access. Retrieved on June 26, 2007 from http://elpub.scix.net/data/works/att/140_elpub2007.content.pdf

Lamb, B. (2004). Wide open spaces: Wikis, ready or not. EDUCAUSE Review, 39(5) (September/October), 36-48. Retrieved on June 23, 2007 from http://www.educause.edu/pub/er/erm04/erm0452.asp?bhcp=1

Larsen, K. & Vincent-Lancrin, S. (2005). The impact of ICT on tertiary education: Advances and promises. Washington: OECD/NSF/U http://advancingknowledge.com/drafts/Larsen-The%20impact%20of%20ICT%20on%20tertiary%20education%20-%20AKKE.doc

Neus, A. (2001). Managing Information Quality in Virtual Communities of Practice. Retrieved on May 24, 2007 from http://pascal.case.unibz.it/retrieve/3266/neus.pdf

Open Source. (2007, July 21). In Wikipedia, The Free Encyclopedia. Retrieved July 21, 2007, from http://en.wikipedia.org/wiki/Open_source

Parker, K. & Chao, J. (2006). Wiki as a Teaching Tool. Interdisciplinary Journal of Knowledge and Learning Objects 2006, 3, 57-72. Retrieved on May 23, 2007 from http://ijklo.org/Volume3/IJKLOv3p057-072Parker284.pdf

Smith, M. A. (1999). Invisible crowds in cyberspace: Mapping the social structure of the Usenet. Retrieved on July 3, 2007 from http://research.microsoft.com/users/masmith/Invisible%20Crowds%20in%20Cyberspace.doc

Stella, A. (2006). Quality Assurance of Cross-border Higher Education. Quality in Higher Education, 12(3), 257-276.

Stvilia, B., Gasser, L., Twidale, M., and Smith, L. (2005). A framework for information quality assessment. Retrieved on May 18, 2007 from http://www.isrl.uiuc.edu/~gasser/papers/stvilia_IQFramework.pdf

Stvilia, B., Twidale, M., Smith, L. and Gasser, L. (2005). Assessing information quality of a community-based encyclopedia. Retrieved on May 18, 2007 from http://www.isrl.uiuc.edu/~stvilia/papers/quantWiki.pdf

Stvilia, B., Twidale, M., Gasser, L., and Smith, L. (2005).Information quality discussions in wikipedia. Retrieved on May 19, 2007 from http://www.isrl.uiuc.edu/~stvilia/papers/qualWiki.pdf

Tam, M. (2001). Measuring quality and performance in higher education. Quality in Higher Education, 7(1), 47-54.

Tapscott, D., & Williams, A. D. (2006). Wikinomics: How Mass Collaboration Changes Everything. New York, NY: Portofolio, Penguin Group.

UNESCO. (2005). Guidelines for Quality Provision in Cross-border Higher Education. Retrieved on May 19, 2007 from http://unesdoc.unesco.org/images/0014/001433/143349e.pdf

Viégas, F., Wattenberg, M. & Kushal, D. (2004). Studying Cooperation and Conflict between Authors with History Flow Visualization. In Proceedings of the 2004 conference on Human factors in computing systems. New York: ACM. Retrieved on June 27, 2007 from http://www2.sims.berkeley.edu/academics/courses/is247/f05/readings/Viegas_HistoryFlow_CHI04.pdf

WikiEducator. (2007, February 19). In WikiEducator, The Free OER. Retrieved 11 March 2007, from http://www.wikieducator.org/WikiEducator:About

Wikiversity. (2007, May 25). In Wikiversity, Introduction. Retrieved on May 22, 2007 from http://en.wikiversity.org/wiki/Wikiversity:Introduction

Wiley, D. (2005). Thoughts from the Hewlett Open Ed Grantees Meeting. Iterating toward openness. http://opencontent.org/blog/archives/192

Yukawa, J. (2005). Story-lines: A case study of online learning using narrative analysis. In T. Koschmann,D. Suthers & T. W. Chan (Eds.), Computer Supported Collaborative Learning 2005: The Next 10 Year (pp. 732-736). Mahwah, NJ: Lawrence Erlbaum Associates. Retrieved July 2, 2007 from http://lilt.ics.hawaii.edu/lilt/papers/2005/Yukawa-CSCL-2005.pdf