Foundation Skills/Developing an assessment/Objectives
|Basics of assessment - principles and tools|
|Developing an assessment||Objectives | Overview of question types | Bloom's taxonomy | Designing an assessment | Communicating expectations to students | Using marking schedules and rubrics | e-Activity | Summary|
- 1 WEEK 12 - Unit 8 Topic 4: Developing an assessment
- 2 Objectives
- 3 Overview of question types
- 4 Web Resources
- 5 Web Resources
- 6 Web Resources
- 7 Web Resources
- 8 Web Resources
- 9 Assessment
- 10 Summary
WEEK 12 - Unit 8 Topic 4: Developing an assessment
Overview of question types
In 1956, Benjamin Bloom developed a taxonomy of thinking skills (cognitive skills). His six categories, in order of increasing cognitive demand, were:
- Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation
In 2001, Anderson and Krathwohl revised Bloom's Taxonomy - their six categories of cognitive skills, once again in order of increasing cognitive demand, are:
- Remembering, Understanding, Applying, Analyzing, Evaluating, Creating
The first three categories are sometimes called the lower order thinking skills (LOTS), whilst the latter three categories are often referred to as the higher order thinking skills (HOTS).
I've developed the following diagram to illustrate the hierarchy in the revised version of Bloom's Taxonomy, together with a sample of action verbs. Action verbs are the critical doing words used in questions. Quite often, the action verb indicates the complexity (level) of the question, e.g. when you ask a student to "justify a method of investing money", the cognitive demand is obviously greater than simply asking the student to "name one way of investing money".
When planning assessment tasks, first revisit the learning outcomes for your course. The action verb used in an assessment question must match the level intended in the learning outcome, e.g. if a learning outcome requires students to "list the tools used to install a wall-mounted flat-screen TV", then appropriate action verbs can include words such as list and name. You wouldn't ask students to "justify the use of tools ...".
Designing an assessment
The method of assessment ( e.g. observation, written theory test, practical report), the types of questions (e.g. multiple choice, computational), and the level of questions (e.g. remembering, creating) that you pose in an assessment task for your students will depend on a number of factors:
- the nature of the learning outcomes for your course (e.g. level according to Bloom's Taxonomy, theoretical or practical)
- the level of your course according to the NZQA framework (e.g. Level 4, Level 7)
- the abilities and experiences of your students at the time of assessment (e.g. basic, advanced)
- the resources available (e.g. space, computers, supervisors, consumables)
Ask yourself the following questions when designing assessment tasks:
- To what extent does the assessment task address graduate attributes as outlined in the graduate profile?
- Does the assessment task match the learning outcome(s) of the course?
- Are all learning outcomes covered by my assessment tasks?
- Is the assessment task reliable, valid, fair, clear, transparent, authentic, sufficient, manageable?
- Is the wording of the questions appropriate to the learning outcomes and the level of the students?
- To what extent do my assessment tasks in the course cater for different learning styles?
- Are students familiar with the method of assessment and the types/level of questions?
- Have students been given opportunity to develop their competence and confidence?
- To what extent do the assessment tasks support the scaffolding of knowledge and skills?
- What evidence of learning is appropriate and sufficient?
- Have students been made aware of your expectations? Do they understand these?
- How will meaningful feedback be provided to students?
- What level of moderation will be used?
Can you think of any additional factors to be aware of when designing assessment tasks? make a list if you can think of any.
If you are new to designing assessments, you might want ot see what your colleagues have done previously (although don't assume that their assessment tasks are perfect!) - ask questions for clarification. The Internet and published books of question banks are both also valuable sources of new assessment questions.
Communicating expectations to students
In Unit 3 of this course, you were introduced to Chickering and Gamson's seven principles of good undergraduate teaching one of which was that good teachers communicate high expectations. Just as you communicate high expectations around course participation, so should you also communicate high expectations in your assessments.
in addition to providing students with written requirements for an assessment task, try to do the following:
- Post an electronic copy of the assessment task online (e.g. in Moodle) together with requirements and expectations
- Have a face-to-face assessment briefing in class to explain the assessment task and answer any questions that students may have
- Have a face-to-face assessment de-briefing in class about a week after the briefing session to clarify requirements and address queries
- Show examples of good, average and poor student work from the past (get permission from students!)
- Get students to judge pieces of work and to justify their decisions
- Provide opportunity for students to develop competence and confidence through practise examples
- Explain what you expect in response to particular action verbs
- Use suitable action verbs in your questions in class or online to develop student thinking
- Set aside time in class for discussions around assessment
- Provide marking schedules and rubrics where possible and exlpain to students how these work
- Educate students to avoid plagiarism
- Design assessment tasks to minimise plagiarism
- Provide useful and timely feedback on student performance andc encourage and advise students on how to do better
Using marking schedules and rubrics
The Collins Online Dictionary defines a marking scheme (or marking schedule) as:
- "(education) a plan or guidelines used in the marking of school children's or students' written work by teaching staff"
The marking scheme gives an indication of the mark allocation for various aspects of an assessment task (usually a task that has a low objectivity of scoring, such as an essay). The use of a marking scheme gives students an indication of what will count for marks, which aspects of the assessment task carry more weight and can also enable students to be more focused. For teachers, the use of a marking scheme increases the reliability of the assessment task. Examples of simple marking schemes can be found on page 2 of the resource Preparing examination questions (essays). The marking scheme does not usually indicate how an assessor will make judgements.
Andrade defines a rubric as:
- "... a scoring tool that lists the criteria for a piece of work, or “what counts” (for example, purpose, organization, details, voice, and mechanics are often what count in a piece of writing); it also articulates gradations of quality for each criterion, from excellent to poor."
So, a rubric, in addition to providing the criteria (or aspects) that count for marks, also indicates how the teacher (assessor) will arrive at a judgement (and a mark) for each criterion (or aspect). See Figure 1 in Andrade's article Understanding Rubrics for an example of a rubric used to grade reports on student inventions. The Andrade article also provides a rationale and tips for creating rubrics.