Otago Polytechnic/Measuring our open education

Work in progress, expect frequent changes. Help and feedback is welcome. See discussion page.

<kaltura-widget kalturaid='j8yhgxy59k' size='L' align='R'/> This is a project to evaluate the use of social media to develop open education and research practices at Otago Polytechnic. When completed (April 2010) it will be brought together into a User Guide for tertiary education institutions to use for benchmarking and further.

Summary

It costs $4000 to train one person how to use social media to source, produce and publish open educational resources for their teaching practice. That person will go on to return $9188 worth of savings and gains to their organisation in the first year. ($4542 per year worth of brand awareness, $1031 per year in quality gains by sampling and reusing free content, and up to $3615 per year worth of savings in infrastructural and support costs by using free media publishing services). Multiply that by 10 staff over 3 years, $275640 is not a small number.

Themes emerging from interviews with Otago Polytechnic staff regarding open education. In the interviews with 5 staff: The rationale for open education was mentioned 109 times, covering 35% of the discussion. The practice for open education was mentioned 95 times covering 29% of the conversation. Resistance to open education was mentioned 54 times, covering 18% of the conversation. Beliefs about teaching and learning were mentioned 41 times, covering 11% of the conversation. Full report of the study here

How this approach to teaching practices impacts on senses of job satisfaction, motivation, and perceptions of teaching and learning performance (values) is yet to be determined. We are waiting for reports from an ethnographic study to be conducted by external researchers. How those values transfer to student motivation and success can be determined by literature review. We hope to be able to use the study on staff values to propose impacts on learning outcomes and student satisfaction, leading on to a further study.

If it is agreed that these returns are tangible and useful, it will be recommended that a committed training regime (one that compensates a teacher for their time in training) be implemented, and that incentives and rewards be put in place for teachers that go on to use popular social media for open educational practices.

Review

• Progress notes from the project coordinator
• Comments and discussion after reporting initial findings.
• Review statement from a leading peer in the field of open education
• Review statement from a leading peer in the field of educational development
• Review statement from a leading peer in the field of market studies and its application to reviewing educational effectiveness

Methods

Three studies were conducted to evaluate the impact, usefulness and/or returns or costs of developing open educational practices using popular social media platforms.

1. An evaluation of usage statistics for educational resources produced by 13 staff,
2. an ethnographic study to determine senses of value from the practices,
3. and an evaluation of 2 courses that trialled open education

• Research plan

Usage

The evaluation of usage based based on statistical evidence was used to determine a basic return on investment calculation. A dollar value on over all gains was determined followed by an estimate on over all savings. From this was deducted the over all costs then divided by the number of people in the convenience sample.

Gains

An equation was developed based on the costs and returns of current brand awareness marketing campaigns (billboads and newspaper), to determine the equivalent returns in brand awareness through open educational resources on popular social media platforms.

Savings

Advice from other institutions running equivalent technical infrastructure was use to establish an estimate of savings in running costs through the use of free social media platforms.

Costs

A review of efforts by the Educational Development Centre to informally train staff in the use of social media for open education was used to establish an estimate for the cost of training as though that training was focused and deliberate.

Break down of evaluation tools

Brand awareness

The worth of return on existing brand awareness marketing efforts, used to calculate the return for open educational resources:

• Take the cost of a 1/4 page ad in a local newspaper (not including design): $629, divided by 47000 copies = 1c per view if one view is equal to 1 copy of the newspaper.
• Take the cost of one billboard: $3490 (including banner but not including design), and divide by 19000 cars passing = 2c per view if one view is equal to one car passing.
• The worth of an open educational resource (in terms of brand awareness only) will be be calculated based on a single view being worth 3c of brand awareness to the Polytechnic.

Quality enhancements

The number of times an educational resource reuses free media and other open educational resources is multiplied by the general worth of such media:

• An image based on a low royalty fee rate: $20
• An audio track based on a low royalty fee rate: $50
• A page of text based on it taking 1 hour to produce: $50 (although no measurement was taken as to how many time such a resource was reused)
• A 5 minute video or 10 slide presentation based on it taking 3 hours to produce: $150 (although no measurement was taken as to how many time such a resource was reused)

Training and resourcing one person

• 10 hours of 1:1 intensive training: $1000 (includes compensation for the trainee's time)
• 10 hours of 1:1 support: $1000 (includes compensation for the trainee's time)
• Equipment: $2000

Hosting

as an equivalent cost if the Polytechnic was to set up and run its own social media platforms:

• Wiki ~NZ$15000 per year in true fixed cost for setting up and running a wiki (based on Wikieducator pricing) and not accounting for the worth of opportunities implicit in the popularity of the service.
• Media ~NZ$20000 per year in true fixed cost for setting up and running a media sharing site based on estimates made at University of Canterbury's UCTV, and not accounting for the worth of opportunities implicit in the popularity of the service.
• Blogging ~NZ$10000 per year in true fixed cost for setting up and running a blogging system based on equivalent utility service provider charges, and not accounting for the worth of opportunities implicit in the popularity of the service.

Results for each social media platform

Wikieducator

In short, $39519 worth of gains, less $15000 in training costs. These 5 staff have made a return of $24519 in the first year. If usage remains constant, they will return $64038 by the 2nd year, and $103557 by the 3rd.

5 staff published over 500 educational resources on Wikieducator. These resources have been viewed over 970620 times, totaling $29119 worth of brand awareness. By using freely licensed images and graphics, these 5 staff have improved the quality of their resources to the worth of $10400. Additionally, all the staff using the free services provided by Wikieducator have collectively saved the Polytechnic around $15000 per year in costs of setting up, running and support services for its own wiki platform.

• Sample size (number of staff): 5
• Cost of training: $20000
• Number of resources: 500
• Total data size: 60 MB
• Total views: 970620
• Nominal worth in terms of brand awareness: $29118.60
• Total number of free images reused: 520
• Nominal worth of quality gains by reusing free images: $10400
• Worth of the wiki service provided by Wikieducator: $15000

Youtube

In short, $11268 worth of gains, less $9000 in training. These 3 staff have made a return worth $2268 in the first year. If usage rates remain constant they will increase their return to $13536 by the 2nd year, and $24804 by the 3rd.

3 staff published over 95 educational resources on Youtube. These resources have been viewed over 355600 times, totaling $10668 worth of brand awareness. 12 of these videos use freely licensed audio tracks worth $600. Additionally, all the staff using the free services of Youtube have collectively saved the Polytechnic $10000 per year in costs of providing that service internally (1/2 of the cost of a total multi media hosting service when divided between Slideshare and Youtube).

• Sample size (number of staff): 3
• Cost of training: $12000
• Number of videos: 95
• Total data size: ~950 MB
• Number of free audio tracks: 12
• Nominal worth of free audio tracks: $600
• Total views: 355600
• Responses: 2433
• Nominal brand awareness worth: $10668
• Worth of the media service provided by Youtube: $10000

Slideshare

In short, $1820 worth of savings and gains, less $6000 in training. These 2 staff are short $4180 from making a return in the first year. Should the usage rate remain constant, they will reduce their shortfall to $2360 in the 2nd year, and $520 by the 3rd year.

2 staff published over 94 educational resources on Slideshare. These resources have been viewed over 14000 times, totaling $420 worth of brand awareness. By using freely licensed images and graphics, these 5 staff have improved the quality of their resources to the worth of $1400. Additionally, all the staff using the free services of Slideshare have collectively saved the Polytechnic $10000 per year in costs of providing that service internally (1/2 of the cost of a total multi media hosting service when divided between Slideshare and Youtube).

• Sample size (number of staff): 2
• Cost of the training: $8000
• Number of presentations: 94
• Total number of free images used: ~70
• Nominal worth of reused free images: $1400
• Total data size: ~188 MB
• Total views: ~14000
• Responses: 8
• Nominal worth in terms of brand awareness: $420
• Worth of the media service provided by Slideshare: $10000

Blogging

In short, $6438 worth of gains, less $9000 cost in training. These 3 staff are short $2562 from making a return. If usage remains constant, they will make a return of $3876 by the second year, and $10314 by the 3rd year.

3 staff published over 102 blog posts relating to the Polytechnic. These resources have been viewed over 181281 times, totaling $5438 worth of brand awareness. By using freely licensed images and graphics, these 3 staff have improved the quality of their resources to the worth of $1000. Additionally, all staff using free blogging services have collectively saved the Polytechnic around $10000 per year in costs of providing that service internally (1/2 of the cost of a total multi media hosting service when divided between Slideshare and Youtube).

• Sample size (number of staff): 3
• Cost of training: $12000
• Number of posts: 102
• Total number of free images used: ~50
• Nominal worth of reused free images: $1000
• Total data size:
• Total views: 181281
• Responses: 130
• Nominal brand awareness worth: $5438.43
• Worth of the blogging service provided by free blogging platforms: $10000

Staff values

University of Otago Report to the Otago Polytechnic: Otago Polytechnic Staff Perspectives on the Practice of Open Education. Prepared by Russell Butson, Shelagh Ferguson, Carla Thomson May 2010

Open course evaluations

TBA

Recommendations

TBA