General feedback and discussions on the proposed survey
Feel free to join the general discussion on developing the survey by clicking on the reply button below.
Start a new discussion / thread for discussions on relevant items in the proposed survey.
I'm not sure I understand the questions related to formal assessment requirements. If the idea is to require all students (credit and noncredit-seeking learners) to be formally assessed, I'm not sure how practice that could be. If you are referring to formal assessment requirements only for credit-seeking students, I'm not sure how that would influence noncredit-seeking students. And by formal assessment requirements, are you referring to an exit exam, summative assessments, quizzes, etc.? In other words, do formal assessment requirements include both formative and summative assessments?
The section of the survey about formal assessments could include a checklist that asks respondents to check those MOOCs that they took for credit, and thus, likely had a formal evaluation. Also this section would benefit from a quick definition of formal evaluation that identifies the purpose, audience, & methodology. Sorry, but that definition should come from someone with more experience than I have.
Personally, the term formal evaluation is vague. Taking a course for credit is one thing. If they are taking the MOOC for credit, we might simply task what types of assessment were involved: informal discussions (i.e., forums, Twitter, etc.), academic prompts (i.e., SQ3R, RAFT, etc.), question and answer (essential questions), task-based learning, problem-based learning (problem setting and solving - action research), eportfolios, performance tasks (Wiggins and McTighe, 2005), quizzes, exams, etc.
Another question might relate to instruments, tactics, or spaces used: mindmaps, rubrics, charts, charts, OERs, personal website, personal blog, wiki, personal learning network development (before and after MOOC),
And finally, perhaps a question related to types of assessment interaction: self, peer, MOOC facilitator(s), other experts, and other novices. The other novices approach intrigues me because teachers could assess a credit-seeking student by having the student assume the teacher's role in teaching someone else something.
Just some thoughts...any other assessment-related considerations?
Hi Ben,
These are great ideas and we will definitely need to incorporate these discussions in the planning of the OERu's.
The OERu logic model has identified two initiatives where this planning and discussion is needed, specifically: Open Assessment Services and Open Credential services.
I don't think we will be able to realistically cover the design of the assessment and credentialing models with this survey -- it this time just useful to think about the potential demand for assessment services and whether this has a perceived impact on participation and completions rates. The data surveyed in CCK08 clearly indicates significantly higher participation rates in the case of for-credit students -- but that obvious.
Hi Keith,
Agreed - -some explanation and description discussing the rational for the assessment related questions is needed.
We certainly can include an item asking if respondents had previously enrolled in a for-credit option. I thought we could extract that data from the institutions that provided for credit options, because they would have that information. However with privacy issues and additional time needed to chase down that info, it would make sense to include a direct question in whether they had previously enrolled for as a for-credit MOOCer.
Hi Ben,
The idea is not to make formative or summative assessment a requirement for MOOC participation or indeed a feature of the MOOC pedagogy.
Presumably there are learners who enrol and participate in MOOCs because they are free and open. The concept of the OERu university is to scale up the for-credit option to learners who may choose to gain formal academic credit for a MOOC course. Typically the numbers of for-credit options are capped and normal student fees apply.
The OERu will be implementing an option for participants who would like to formerly acquire credentialisation after completion of the MOOC or at some date in the future - -we are aiming to provide pathways to for MOOC learning to be formerly recognised.
The idea is that the institutional costs of assessment will be recouped by the participating institutions on a cost recovery basis -- which will be far cheaper than full tuition fees. We estimate in New Zealand that costs of assessment would not be more than around 20% of the cost of full tuition.
The OERu partners have not planned the details for the assessment model yet -- but I expect we will integrate the experience of distance education providers who have been providing assessment and credential services at a distance for a long time. I suspect e-portfolios would also be useful in providing evidence of the learning process.
Any suggestions or ideas on how to improve the formulation of the relevant questions to convey what they are intended to mean?
Thanks Wayne for your explanation. If we want to know how other institutions have assessed credit-seeking students taking a MOOC or MOOCish-type of course, why not just ask the institutions? Start putting together some keystone assessments (and related instruments) that are meaningful and relevant to the student but based on the type of MOOC being considered and go with that.
Assessment gets a little slippery because assessment and instruction are (or should be) so closely integrated - at least they will be in most MOOCs. Learners oftentimes fail to distinguish the difference between testing, evaluation, and assessment, plus they might be influenced (positively or negatively) by any of these notions based on a prior experience with a teacher. For this reason, I'm not sure if asking the learners about assessment will get the intended results (I could be wrong.).
Instead of using words like assessment, evaluation, or testing, let's just ask them if they've ever used a blog, wiki, eportfolio, etc.
If we create a questionnaire for institutions, that would be a different story.
Any other thoughts?
Hi Ben,
Absolutely - -the OERu anchor partners will definitely be speaking with the institutions who have presented MOOCs with for-credit options and find out what assessment models and approaches were used.
I also agree that valid and reliable assessment is a complex area and is closely related and part of the overall pedagogy deployed.
I agree with your sentiments by asking folk if they've ever used a blog, wiki or portfolio and am planning to include a question in this survey relating to usage / experience with this media.
Lots of work still needed in planning the OERu -- but on the positive side we are doing this and currently based on our anchor partners we will be able to provide accrediting options for OER learners in New Zealand, Australia, Canada and the US at this time.
