Feedback on Questionnaire 1

From WikiEducator
Jump to: navigation, search


Back to TOUCANS Research instruction page



Icon key points.gif
Key points
  1. You must be logged into WikiEducator to provide feedback in the wiki. (Anyone can create an account on WikiEducator.)
  2. Consult the draft items in Questionnaire 1
  3. Provide your feedback and suggestions for improvement on Questionnaire 1
  4. Consult the draft items in Questionnaire 2
  5. Provide your feedback and suggestions for improvement on Questionnaire 2.



General feedback (relevant to both questionnaires)

  • Pleased to see that you have specified a CC-BY license for publications. However, you need a statement to clear the IP of the data submitted by respondents. I would request something like: "As an open research project you agree to dedicated your responses to the public domain. The data collected from this research project will be made available with personal information removed as open data for reuse by the open research community.--Wayne Mackintosh 02:19, 4 May 2012 (UTC)
  • Questionnaire 1 is intended for OERu anchor partners. You need to think about how you will manage submissions from non-OERu and non-UK respondents. I suggest using a 3rd questionnaire for the WikiEducator and open community. --Wayne Mackintosh 02:19, 4 May 2012 (UTC)
  • If you opt for a 3rd questionnaire, collecting regional or country data would be an advantage. Think about ease of aggregating this information. Perhaps using the World Bank or UNESCO global regional classifications. --Wayne Mackintosh 02:19, 4 May 2012 (UTC)
  • Include a confirmation question on whether the respondent would like to be attributed --- some respondents may not wish to be attributed for their contributions.
  • Make sure that it is easy to remove personal information in the final Google spreadsheet when making the open data available. --Wayne Mackintosh 02:23, 4 May 2012 (UTC)
  • Include a warning that if personal identifying information is included in the open responses, you cannot guarantee anonymity or you will need to clean this up before publishing open data. --Wayne Mackintosh 02:23, 4 May 2012 (UTC)
  • There are differences between OERTen as the network implementing OERu and the OERu itself, but would recommend using OERu or OERu network though out to avoid confusion with respondents outside the network who may not know the difference. --Wayne Mackintosh 04:42, 4 May 2012 (UTC)
  • I assume its OK for more than one respondent per OERu partner institution to respond. In this case, I suggest a note that clarifying that they are responding as an "informed" OERu partner staff member, but to clarify that their responses do not necessarily represent institution-wide policy. --Wayne Mackintosh 05:13, 4 May 2012 (UTC)

Suggestions for additional questions

It is important to keep the questionnaire short -- however, this is a unique opportunity to gain comparative information -- I would strongly recommend the inclusion of an item with appropriate Likert scale to gauge the reasons why OERTen members have joined the network and how these factors relate to each other. (In the case of non-anchor partner questionnaires, you could say something like, Rate the influence that the following factors would have in leading to a positive decision were your institution to join the OERten (You can work on improved wording here - -you know what I mean.). This would provide an opportunity to map and compare strategic and operational educational objectives between OERTen partners and non-partners --Wayne Mackintosh 02:36, 4 May 2012 (UTC)

  • For example, Rate the relative importance of the following factors regarding your institution's decision in joining the OERTen:
    • Philanthropic, i.e. widening access to more affordable education opportunities for learners excluded from the post-secondary system as a community service.
    • Potential to reduce cost and save time associated with the development of learning materials
    • Participation in an international network of accredited institutions
    • Low risk innovation strategy
    • Opportunity to increase local student enrollment from OERu learners who decide to complete their degree studies at my institution.
    • International marketing of my institution
    • Opportunity to diversify revenue streams by incorporating value-added services (in addition to assessment and credential services) to the OERu model
    • Testing the OER model as a low risk project in a rapidly changing market
    • Using the OERu network as a learning environment for improving teaching practice in online learning at my institution.
    • Retain competitive advantage as OER becomes more mainstream.
    • Etc list other reasons obtained from your research
    • Other
  • It would be extremely useful to gauge the opinion on anticipated conversion rates of the ration of learners participating in free OERu courses and requests for formal assessment services. I would calibrate in small increments of 5%, Eg less than 5% paying for formal assessment services going up to say 30% and then increments of 10 form 30 to 50% and then jumping in bigger increments. On second thoughts, it may be better to ask the average conversion rate in multiples of 5. --Wayne Mackintosh 04:36, 4 May 2012 (UTC)
  • The other question worth asking imho, is an estimate of the cost of assessment services expressed as a percentage of full-tution costs. This could be broad ranges starting from less than 15% and in multiples of 5 up to 50% and then last category being greater than 50%.

Accreditation and credit transfer question

  • The stem of the accreditation and credit transfer question does not match what I think you are trying to etablish. --Wayne Mackintosh 02:54, 4 May 2012 (UTC)
  • There are two issues here: 1) one relates to partners recognizing credits within the network -- irrespective of the total number of credits that will be accepted locally. and 2) The other relates to the number of course credits -- You need to be careful when clarifying these issues -- it may need two separate questions. --Wayne Mackintosh 03:11, 4 May 2012 (UTC)
  • Consider the following revised stem. Credits obtained from another institution for an OERu course may be counted towards a credential (eg certificate, diploma or degree) within the parameters of my institution's assessment and credentialing policies and OERu partners are aiming to facilitate credit transfer of locally approved OERu courses. In your view, how important is it for the partners to participate in OERu transfer credit arrangements for approved courses (assuming these fall within local policy parameters for the number of credits which can be transferred). -- Needs a little tidying up, but you will get the idea. In other words - -this question focuses on the number of partners participating in credit transfer rather than the number of credits which will be accepted towards a degree.
  • I also suggest tweaking the distractors to make this distinction clearer, for example: If all OERu partners implemented credit transfer within local policy parameters, it would have a positive impact on the success of the OERu -- Alternatively: The number of OERu partners participating in credit transfer (within local policy parameters) would have a POSITIVE impact on the success of the OERu. Use bold or capitals for the distinguishing element in the distractor --Wayne Mackintosh 03:29, 4 May 2012 (UTC)
  • Simply the 2nd distractor use no difference rather than no significant difference which is a statistical concept we have not tested. --Wayne Mackintosh 03:33, 4 May 2012 (UTC)
  • To deal with the 2nd issue of number of credits, consider the following: Indicate the extent you agree with the statement: OERu partners who can accommodate a higher percentage of course transfer credits towards a credential will have competitive advantage above those who accept a lower percentage of course transfer credits. --- Add an appropriate Likert scale. --Wayne Mackintosh 03:37, 4 May 2012 (UTC)

Curriculum question

  • The question is not aligned with the OERu model -- Institutions commit to contributing 2 courses, there is is no requirement to release all courses as OER. This question is misleading :-(. --Wayne Mackintosh 03:41, 4 May 2012 (UTC)
  • I think you are asking about the best approach for developing OERu courses comparing: 1) Developing or releasing the OERu course contribution under an open license (eg -- single institution development model); 2) A remix or assembly model building courses from existing OERs; and/or 3) Collaborative development of OERu courses where institutions share the development responsibility. --Wayne Mackintosh 03:45, 4 May 2012 (UTC)
  • In this question the rating scale is confusing - -it is conflating too many issues, that is the number partner institutions doing something with the things they are doing. It's mixing questions. Why not ask, which of the following approaches is likely to contribute to greater success of the OERu network with a simpler likert scale? --Wayne Mackintosh 03:47, 4 May 2012 (UTC)
  • Important that we flag that transfer relates to "cognate" courses; a programme of study involves a coherent selection of courses studied in a logical sequence not a shopping basket of discrete items. (Tony)

Assessment question

  • The title does not convey what the question is asking -- this is about assessment and credentialing protocols for the OERu - -consider revising the heading.
  • Again -- I don't think scale is working for this question. The number of institutions who do this, is not necessarily an indicator of what you're trying to establish imho. --Wayne Mackintosh 03:51, 4 May 2012 (UTC)
  • The question is also problematic because it's comparing apples with pears. PLAR is typically used for learning acquired outside of the classroom setting -- simply stated, formal accreditation for life experience. This is something very different from course-based credits. Many OERu learners will be following "courses" mapped to course outcomes which is not "true" PLAR. --Wayne Mackintosh 04:10, 4 May 2012 (UTC)
  • The inclusion of standardised exams complicates the question -- if the institution accepts credit for standardised exams, and these could be done cheaper through the external examination providers, then OERu learners should use this model. It would be better to strip this out from the question and simply ask, does your institution accept standardised exam results for credit at your institution (eg CLEP). You can easily establish the cost of these services. --Wayne Mackintosh 04:10, 4 May 2012 (UTC)
  • I would revise this question and divide into two sub-questions. 1) Which of the following methods will your institution use for providing formal credit to OERu learners - -list the options 2) A question asking about the most likely method for assessment in the OERu network for the following options. --Wayne Mackintosh 04:10, 4 May 2012 (UTC) The options are:
    • PLAR
    • Challenge for credit or proctored examination
    • Course-based portfolio
    • Automated online assessment
    • Work-based learning
    • Standardised third-party examinations

Learner support

  • In its current format - -this is not a valid question. The OERu model does not aim to provide institution-based learner support. I think the question should rather focus on the importance of Academic Volunteers International in contributing to successful learning in the OERu model. --Wayne Mackintosh 04:17, 4 May 2012 (UTC)
  • The OERu model will be making extensive use of social media technologies irrespective of the current use at member institutions. The core technologies for hosting OERu courses and interactions are available on the open web or will be hosted by the OER Foundation -- So as the question stands, its not going to add much value. I suggest an item asking about the importance or contribution of social media to successful learning in the OERu model would be a better question. --Wayne Mackintosh 04:20, 4 May 2012 (UTC)
  • Again, the extent to which OERu partners currently use learning analytics or volunteer support in their existing model is not relevant to the future of OERu. What I think we should be asking is the opinion of OERu anchor partners on the relative importance or contribution of these strategies towards successful OERu learning. --Wayne Mackintosh 04:23, 4 May 2012 (UTC)
  • I don't think the work-based learning question fits well here. --Wayne Mackintosh 04:23, 4 May 2012 (UTC)

Other comments

These comments refer to the OERTen partners question:

Question 8: I'm wondering if "All or most" would work better than only "All" because the latter seems to slant the question towards the unity of the partnership rather than to the idea that the weight of the project is moving in that direction.

Questions 9-10: It might be clearer if “all institutions” is stated in the sub-questions rather than in the main ones.

Good luck with this project Gabi! --Irwin DeVries 22:08, 9 May 2012 (UTC)

Retrieved from "https://wikieducator.org/index.php?title=OERu/Planning/TOUCANS_Research_questionnaire_consultation/Feedback_on_Questionnaire_1&oldid=906284"