Introduction to Research Methods In Psychology/Meetings
Contents
March 2016
Gail: Farhad has made a comprehensive final exam and matching practice exam for the OURu version of course. Assessments in the OERu version will only be formative, not summative, so students will work through course and then approach an OERu institution to write exam for credits (small charge to cover costs of administering exam). Gail and Irwin to discuss who might adminiter the final OERu exam for OERu...TRU or some other institution??
--------------------------------------------------------------------------
November 2004 Discussion about Assignment 2
Gail: The various audiences we are designing the course for:
Students taking course for own learning with NO instructor (many OERu learners will be independent learners)
Students taking course with an instructor but not a TRU-OL instructor
Students taking the OL version of the course with a TRU-OL instructor
Some students will try assignments for own learning without credit, and others will submit assignments for grading since they want credit.
The original Assignment 2 Part D is problematic since it assumes learners are part of a cohort with other learners. We have to come up with another version of Part D that speaks directly to independent learners who may not have knowledgeable peers to consult…what can/should they do? Find out about the Peer Review project or tool that OERu was working on Summer 2014.
-----------------------------------------------------------------------
Rajiv: Farhad and Rajiv discussed the assignments at length and agreed that the current Assn 4 can be converted to a formative activity for students for that Unit. The 7.5% released will be redistributed such that Assn 1 is now worth 15% and Assn 2 is now worth 10%. The other two assignments will remain unchanged. They also spent quite a bit of time thinking about Assn 2 and how to rework it so that it meets the needs of all 3 audiences. In the end they both felt that the best approach would be to reconfigure Part A and Part B so that each has 5 options and that students are randomly assigned one for each part (this gives us 25 possible combinations in all). Part A will focus on analyses of media reports of non experimental studies (Rajiv will choose these) and Part B will include five different sets of scores for the same variables.
___________________________________________
Gail on 12-11-2014: Hmmm...so all students see all 5 options for Part A and all 5 options for Part B out there on the open internet? And if there is an instructor, that person will tell a student the random combination to submit for AS 2...that plan works OK for students with an instructor from start but still exposes all the "gradable questions" with one right answer to everyone. And there is no person to randomize the options for the many students with no instructor at all...or have you figured out a way to randomize the options for the group with no instructor?
Suggestion: what about showing everyone just ONE option in each part of AS 2 to do as a formative assessment with answers provided on a separate page so ok for everyone to see...and the learning takes place during the course when you want it to happen. Then keep the other four (or less) options for use ONLY by credentialing OERu institutions and by TRU-OL...still to be given in random fashion method if desired which helps to ensure AS 2 evaluation is valid and reliable.
So all students get some assignments up front in open course (but only the formative AS 2 would be in open course and not the summative randomizable AS 2 options)
OERu students who want credit at end have to pay for an evaluator, submit non-problematic assignments related to research, submit a randomized AS 2 they are given after paying fee and then complete the invigilated final exam to get credit (the randomized AS2 could even be part of the final exam if they don't need a lot of time to do it or you can give them some time to complete and submit it for marking)
Those students with an instructor from start will have to do all assignments during course in order as originally planned, including randomized AS 2 instructor gives them or whatever AS 2 they are given in their course in private LMS, and then do their final exam at end of course.
-------------------------------------------------------------------
Rajiv: I think you have it absolutely correct. The completely open version would not list all of the options. It would be relatively simple to create the fully open (formative) option with the answers on a different page.
As you note, OERu students who want credit and who will pay a member institution can submit their research assignments (1, 3, & 4) for grading along with their randomly assigned version of Assignment 2 and their final exam.
-----------------------------------------------------------------
Perfect! Great team work! Together,w e have figured out a great solution for assignments that have “right answers” that other OERu institutions can learn from. Making the majority of assignments authentic and unique is the best way to go, but this method should work fine for the AS 2 and others like it.
My only question is how long will it take a student to do the randomized AS 2…can a random version be added first to each final exam and done in an invigilated setting? That way, only one final completed knowledge-based exam is submitted to a qualified OERu marker for credit, along with the unique research assignments. Or is there a reason that it must be separate from the final exam? Would you say that a student must first submit ALL assignments before they are allowed to do a final exam?
Please update our detailed course plan (that we were calling the blueprint before) with new assessment weighting and with the plan for Assignment 2 (so others can learn from our discussions).
----------------------------------------------------------------
November 10, 2014
Rajiv, Gail (and later Carolyn in Production)
Question asked by Rajiv: For Assignments 2 & 4, there are specific answers to the questions posed and so posting these online will compromise the integrity of the course assessment. How should we handle this?
Initial answer from Gail: One idea is to think of variations of those two assignments...the OL version could be slightly different from the OERu version. Another idea is to somehow make any assignment questions that have one right answer NOT count for marks i.e. give them to students as practice still but NOT as part of the assignment for marks...maybe put those questions in your final exam. OERu students could be encouraged to do all of the assignments/learning activities you put online but getting credit in the end might have to rely more heavily on a different final evaluation package that is put together by a OERu-credentialling institution....such as TRU.
Brainstorm: Move Assignment 4 questions into final exam since they deal with experimental info and redistribute the marks among four assignments. Rajiv and Farhad can think some more about the wording of the Assignment 2 questions...might have another way to get at same info from students. Maybe make Assignment 2 more generic so that students find a non-experimental study themselves from a suggested source and answer given questions about flaws etc. No key should be given to students with correct answers but a marking scheme could be shared with students.
Production did a bit of searching and there are some basic tools for giving dynamic feedback (actually more than initially thought). These tools could be employed in a self-assessment scenario for students to test themselves. The answers are held within page so not difficult to actually see if a student wanted to.
Example pages:
http://wikieducator.org/Maintain_a_chainsaw/Activities/multi_choice_quiz
http://wikieducator.org/Open_Content_Licensing/Example:_quiz_taster (true/false) http://wikieducator.org/Reusable_OER_quiz_example (multiple choice and Cloze questions)
A non-dynamic version of self-test is given on following page:
In this case, answers are provided a separate page. This is a technique OL uses in Blackboard as assessment tools can only really self-grade the most mechanical of question types.
This link will give a listing of different templates. The listing the the /doc are examples of template in use.
[[1]]
Oct 28, 2014
Rajiv and Gail discussed Assignments 2 and 4 by phone. Although we are using an open text and various OER in course, Rajiv is having problems finding open examples of various research designs for students to review and critique. He will link out to some he found on the Web. Gail explained that any materials being embedded in our open course has to be OER, but it's OK to allow students to link to free, appropriate, authentic resources in open journals and the Web, if necessary...especially if it is a research course and doing external research is part of the outcomes of the course.
Rajiv is working on all assignments before starting work on all modules (using his teacher voice).
Oct. 9, 2014
We are keeping PYSC 2111 as a 3-credit course (not three 1-credit courses as discussed). The writers thought that Research Methods is very much a topic that requires cumulative and thoughtfully-sequenced learning. If anyone wishes to create three 1-credit chunks (3 mini mOOCs)....a tentative division might look something like this:
- Micro 1: Introduction to Research Psychology, Theories and Measurement (Units 1 and 2)
- Micro 2: Non-experimental and Experimental Research Methods (Units 3 and 4)
- Micro 3: Data analysis & Reporting (Inclusive of research report) (Unit 5)
Oct. 1, 2014
Attendees and Location
Gail Morong, ID
Production: Carolyn Teare, Cory Stumpf, Thomas Sandhoff
Dr. Rajiv Jhangiani (main course developer) (via phone)
OL 346
Notes
- Course Blueprint is complete and therefore we need to address structure of course
- Idea brought forth by Wayne M to create 3 mini-courses
- Discussed if this is feasible/desired as it would directly impact page structures (naming of pages) and it is much easier to do correctly than to fix. It is also much easier to bring together components than separate.
- Phoned Rajiv to discuss: implications for course material/development.
- Rajiv will consult with Dr. Farhad Dastur (consultant) and get his input over a tentative 3 part structure. Some rethinking on assessments might be required but general agreement that there are ways to make it work.
- Production will refrain from page creation until confirmation of overall structure. Production would need correct titles of each section before commencing page structure.
- Production will follow and use the templates provided by Wikieducator. Initial page setup will include top and bottom nav components.
- Subject Matter Expert will reference the within page formatting templates provided with Wikieducator either by direct use or by creating production notes.
- Production can go in after and provide page cleanup which could include adding addtional within page formatting templates, fixing file linkages (image, documents, external resources)
- [Pedagogical Template] should be reference document used for within page templates
Aug. 13, 2014
Attendees and Location
Gail Morong, ID
Irwin Devries and Naomi Cloutier
Production: Carolyn Teare, Sean Maurico, Cory Stumpf, Thomas Sandhoff
OL 344
Notes
- The PSYC 2111 project is TRU's 2nd committed offering of 3 to OURu. We want to offer open course to OERu but also use it as one of TRU-OL's courses with an Open Learning Faculty Member (OLFM).
- Radical transparency is very important in open development...all meeting notes and course team discussions will be kept on this page in wiki.
- LMS options for delivery of course a) Build course in wiki and use the wiki as the sole LMS for both audeinces b) Build course in wiki and import it into Blackboard Learn using iframes (when you edit in wiki, all edits transfer over.), or just link out to pages in the wiki. OL version can have slight modifications to suit OL standard web audience- most of course might be identical with a few extra in-house pages where needed. OLFMs have to be up to speed with LMS used. Note to Production: the Technology team are experimenting with a "live" snapshot of content that works in WordPress - worth checking out.
- Wiki has revision history, but time consuming to find a specific past instance of a page. PDF of course can be made, if needed.
- Assessments have to be designed with openness in mind.
- We'll have to decide how we want text and other readings to be included in course...maybe as chunked PDFs
- This course is based on an open textbook.
- Question about open texts - do old versions get archived? We noticed the chosen open text is Canadian...will it work for students all over the world? Gail to discuss with SMEs
- Production folks could create own accounts and play around in the sandbox - they have already been doing some experimenting with wikitext
- Gail to set up session with Scott in IPO re: Creative Commons licences...images to be uploaded first into Wikimedia Commons before going in course
- There is a course sprint going on to develop a course for teaching new OERu developers wiki and other needed skills...anyone free to participate
- Production interested in a) getting content into wiki..will experiment with making "shells for SMEs once blueprint is complete b) getting content out of courses
- Meeting needed in 3 weeks to discuss progress of project
Aug 1, 2014
Attendees and Location
Gail Morong, ID
Denise Weisgerber, F2F Faculty
Dr. Rajiv Jhangiani (main course developer)
Dr. Farhad Dastur (consultant)
OL 329 and Skype
Notes
This planning session only focussed on learning outcomes. We aligned the F2F and OL outcomes and came up with a list that works for an intro course to research methods that is geared mainly for psychology students but could work for any social science.
Action: Denise to share outcomes with her F2F colleagues for feedback.
Action: Rahad and Rajiv to brainstorm the assessments to align witht the outcomes and complete the table in blueprint. Also work on topics and organization of modules.
July 29, 2014
Attendees and Location
Gail Morong, ID
Mike Looney, Program Coordinator for Arts
Denise Weisgerber, F2F Faculty
OL 340
Notes
1. F2F has a 3rd year research course PSYC 3190…so we have to make sure our 2nd year course does not overlap with that one e.g. no serious stats needed in 2nd year:
PSYC 3190
Experimental Design and Quantitative Methods
This course provides an integrated presentation of the methods, principles, and ethics of psychological research and explores the statistical techniques utilized for the analysis of these data. Instruction in the formal reporting of psychological studies is also covered.: PSYC 3190L
Prerequisite: PSYC 1110/1210, PSYC 2100 and PSYC 2110
Corequisite: PSYC 2110
2. In Sept 2014, Mike will have to change our course title, calendar description, outcomes, assessments, prereqs, etc...once he gets everything from Gail
3. After reviewing the tips on writing outcomes Gail sent the team, Denise is updating her learning outcomes and will share them with others...she hopes to chat with Rajiv and Farhad about them on Aug 1 at 10:45am.
4. Final exam should stay at 40%.
July 21, 2014
Learning outcomes
Gail contacted Arts OL program coordinator, Mike Looney, who said we will be using F2F course description and name but we need to talk to F2F folks about final learning outcomes that both F2F and OL will use. Since EPC document has no measurable outcomes listed, we can use the OL outcomes as a start. Gail invited F2F faculty, Denise Weisgerber, to meet with us week of July 29th and Mike will join us..
July 16, 2014
New Members to Team
Note from Irwin: A few other TRU-OL folks are joining the development team. These are Brian Lamb, Director of Innovation and Carolyn Teare, Production Supervisor. Brian is working with the OERu to plan and develop technologies for content creation (faculty and students), collaboration, syndication, etc. along with improved wiki interfaces and can also provide many ideas for and examples of non-traditional ways to deliver online learning in open environments. Carolyn is an expert LMS wrangler and knowledgeable in document languages (among other things). She oversees the OL course production team, which will provide technical support for the PSYC2111 development team.
July 15, 2014
Attendees and Location
- Instructional Designer: Gail Morong
- Dr. Rajiv Jhangiani (main course developer)
- Dr. Farhad Dastur (consultant)
- Dr. Irwin DeVries
OL 329 and Skype
Notes
Scope Project different from our usual TRU-OL projects since we will be developing in the open using WikiEducator as our LMS, not Blackboard Learn (TRU-OL will import course into our own LMS for our own use after it's completed). Aim is to share our work and invite feedback from larger OERu community as early as possible. The task is to create an online version of the TRU F2F course with same course description and course name...we will delete for "experimental" psychology from our current course name and make a more general research methods course.Our course will be called PSYC 2111: Introduction to Research Methods in Psychology." We will use as much of the current OL course as is useful and applicable and update it.
Resources
We will use an open Psyc text that Rajiv has co-authored as part of a BC Campus open textbook project. Advantages of using an open text are 1) no cost to students and 2) easily adaptable to suit our needs. We will incorporate other existing open websites and materials. These may include materials from the NOBA Project, Social Research Methods Knowledge Base website, TCPS tutorial, and the Research Randomizer. OL Media team can get involved, but only if deemed necessary.
Technology
WikiEducator course pages will be set up for course materials and also to share planning notes. Irwin to share resources and act as mentor for basic wiki editing skills. Suggestion: basic conventions for working collaboratively in a wiki will be helpful. The team will use other openly-available, free communication tools to plan and develop course. OL Production, OERu and WikiEducator community can help with technology needs for course learning activities and assessments as they arise (many tools in BB do not yet exist in WikiEducator...work in progress on a peer evaluation tool, WE has WEnotes microblog tool, but students can use own blog, Twitter etc and posts will get syndicated into course wiki). Gail to give Farhad access to current course in Blackboard.
Assessments
Make as authentic and useful as possible. Students will be encouraged to contribute and add knowledge to course i.e. living ecosystem. Pathway to credit important component ...how to design assessments for open audience? (Credentialling institutions are free to make up own final exams but we must have our own as part of this project). What about digital badge for non-credit students?
Open Design Considerations
Multiusers, so flexibility needed. See course guide in OERu ART 100 for "Ways to Engage" section. Also think of cultural diversity in course..global audience...is research value free? Good debate item...
Irwin mention microcourses, mOOCs...can we make 3 small 1-credit courses instead of one 3-credit one...may not be possible but will consider as we plan. Challenge is to have assessments match each minicourse...some content/themes thread throughout whole course, later parts build on earlier parts
Actions
- Gail to start off notes in wiki and start blueprint with F2F and some OL info. DONE --Gail Morong 19:02, 21 July 2014 (UTC)G Morong
- Irwin to send wiki editing info DONE--Irwin DeVries 15:27, 17 July 2014 (UTC)
- SMES to practice using wiki editing tools and edit Gail's notes. Start planning open course with each other.
