Quality Assurance in Multimedia Learning Materials/quality indicators - product

Quality indicators for Product Evauation

	Section A: Information on MLM			Comments
	A1	Reference Code:
	A2	Module Title:
	A3	Subject Area:
	A4	Coverage:
	A5	Keywords:
	A6	Language:
A7 Target audience:			Pre-school
			Primary School
			Lower Secondary School
			Upper Secondary School
			University
			Post Graduate
			General Public
			Special Needs (specify: ________)
			Others (specify:e.g.'Farmers_____)

 

A8 Min. Hardware Requirements		RAM Stated
Stated:		Processor Stated
		HDD Stated
		Multimedia Requirements Stated
		(e.g. CD/DVD, Speakers, Mic, etc.)
		Internet Speed Stated
		Others Requirements Stated
		(Specify: ______________)
		Not Stated
A9 Min. Software Requirements		OS Stated (e.g. windows version)
Stated:		Browsers Version/Type
		Plug-in software stated
		Other software requirements
		(Specify: _______________)
		Not Stated
A10 Context of Usage:		Support Self Learning
		Support Facilitated Learning
		Support Individual Learning
		Support Group Learning
		Support Formal Learning
		Support Informal Learning
		Others (Specify: ____________)

		Not Stated
A11 Warranty		Warranty provided
		No warranty is available

Section B: Assessment of MLM
			ANALYSIS
CONTENT
No Quality Indicator Descriptor					Quality Measures
B1	Desired learning	1 = Main LO. not stated		1	2	3	4	5
outcomes (LO) are clearly stated.		2 = Main LO. stated but not
		clear
		3	= Main LO. is clear
		4	= Main LO and sub-LO are
		stated
		5	= Main LO and sub-LO are
		very clearly stated.
B2	Content meets objectives.	1 = Deviates from the objs.		1	2	3	4	5
		2	= Partially met
		3	= Meet key objs.
		4	= Meet all objs.
		5	= Meets all obj. and more
B3	Coverage/scope of the	1 = Totally insufficient.		1	2	3	4	5
content is sufficient.		2 = Partially sufficient
		3	= Meet min. requirement
		4	= More than sufficient
		5	= More than sufficient with
		additional activities
B4	Content is reliable.	1 = No user reference site is		1	2	3	4	5
		available
		2	= 1-3 user reference sites are
		listed
		3	= 4-6 user reference sites are
		listed
		4	= 7-9 user reference sites are
		listed
		5	= More than 9 reference sites
		are listed
B5	Content is original	1 = Content infringe copyright		1	2	3	4	5
	(Copyright law).	law
		2	= Content is not original
		3	= Content follows fair-use
		policy
		4	= Content is original
		5	= Content is original and
		unique

LEARNER
B6	Pre-Requisite are stated.	1 = Not stated		1	2	3	4	5
		2	= Stated but not clear
		3	= Stated and clear
		4	= Stated in detail & clear
		5	= Stated in detail, clearly &
		evaluated (pre-test)
B7	Rating of the overall	1 = Very Poor		1	2	3	4	5
	learning experience of the	2 = Poor
learners.		3	= Average
		4	= Good
		5	= Excellent
			DESIGN
INSTRUCTIONAL DESIGN APPLICATION
B8	The content is interactive.	1 = No interaction		1	2	3	4	5
		2	= Limited interactions
		provided
		3	= Some interaction provided
		4	= Many interactions provided
		5	= many interesting
		interactions provided (e.g.
		Simulation and game-based
		learning)
B9	The interface is user-	1 = Interface is totally not user-		1	2	3	4	5
friendly (i.e. familiar & consistent).		friendly (i.e. user need to use
		manual extensively or require
		many hours of training)
		2	= Some parts of interface is
		user-friendly
		3	= Interface is generally user-
		friendly
		4	= Interface is user-friendly
		5	= Interface is very user-
		friendly (i.e. users need not use
		manual or require training)

B10	The MLM promotes	1 = Promotes individual		1	2	3	4	5
	collaborative learning.	Learning
		2	= Some interaction with
		instructor.
		3	= Interaction with peer/instr.
		4	= Group activities are
		provided
		5	= Many group activities are
		provided with opportunity for
		knowledge construction.
B11	The instructional Design	1 = Not effective		1	2	3	4	5
Strategies are effective.		2 = Has some impact on
		learner
		3	= Average impact on learner
		4	= Effective
		5	= Very effective
B12	The MLM has good	1 = Content is boring		1	2	3	4	5
aesthetics values. (The results of good		2 = Some of the content are
aesthetics is		boring
engagement/interest)		3 = Content is generally
		interesting
		4	= Content is interesting
		5	= Content creates high level
		of interest & engagement
B13	Different learning	1 = No specific learning		1	2	3	4	5
styles/approaches are used (diversity).		approach is used
		2 = Only one learning approach
		is used
		3	= 2-3 different learning
		approaches are used
		4	= Few different learning
		approaches are used
		5	= Many different learning
		approaches are used.
B14	The use of technology is	1 = Wrong technology is		1	2	3	4	5
appropriate.		applied
		2	= Some mis-match in
		technology
		3	= Tech. applied is generally
		appropriate
		4	= Tech. applied is appropriate
		5	= Tech. is very appropriate
EVALUATION STRATEGY
B15	Assessment is included.	1 = No assessment conducted		1	2	3	4	5
		2	= Assessment covers only
		some of the learning outcomes
		3	= Assessment (lower level of
		Blooms) covers all learning
		outcomes.

		4	= Assessment (mid-level of
		Blooms) covers all learning
		outcomes with feedback
		5	= Excellent assessment (i.e
		higher level of Blooms) covering
		all learning outcomes with
		constructive feedback.
TECHNICAL CONFORMANCE
B16	Degree of conformance	1 = Does not comply		1	2	3	4	5
to SCORM Standards.		2 = Does not comply but SCO
		concept is used
		3	= Comply with min.
		requirement (i.e. LMSInitialise
		& LMSFinish) & packaging
		4	= Comply & used few data
		model & packaging
		5	= Comply and uses many
		data model & packaging
B17	Degree of conformance	1 = Does not conform		1	2	3	4	5
to Web Standards.		2 = Partial conformance
		3	= Generally conform
		4	= Strictly conforms but
		adopts older version (i.e Web
		1.0)
		5	= Strictly conforms and
		adopts current version (i.e.
		Web 2.0)
B18	Degree of adoption to	1 = Content is strictly		1	2	3	4	5
Open Standards.		proprietary (i.e. use specific
		software to operate)
		2	= Content is strictly
		proprietary but operates on
		open tools (e.g. std. browser)
		3	= Generally adopts Open
		Standards (e.g. client adopts
		open standards but not server)
		4	= Strictly adopts Open
		Standards (e.g. client and
		server adopts Open Standards)
		5	= Strictly adopts Open
		Standards and have free access
		to source files.

			DEVELOPMENT
MULTIMEDIA CONTENT
B19	The use of fonts and	1 = Not readable		1	2	3	4	5
color are appropriate.		2 = Readable but create stress on
		eyes
		3	= Readable
		4	= Readable and consistent
		5	= Readable consistent and
		customizable (i.e. fonts/colours can
		be changed by users)
B20	Language is	1 = Totally not appropriate (i.e. Too		1	2	3	4	5
appropriate to target audience.		high level, too many mistakes,
		heavy accent, etc )
		2	= Not Appropriate (High level,
		some mistakes, some accent, etc)
		3	= Acceptable (Average level, few
		mistakes, no accent, etc)
		4	= Appropriate (simple, minor
		mistakes, clear voice-over, etc.)
		5	= Very appropriate (simple, no
		mistakes, very clear voice-over,
		etc.)
B21	Easy to understand	1 = Diff to understand		1	2	3	4	5
the content.		2 = Some are diff to understand
		3	= Able to understand
		4	= Easy to understand
		5	= Easy to understand & related to
		real world examples
B22	Content is accurate	1 = Mostly Misrepresented		1	2	3	4	5
and factual.		2 = Some misrepresentation
		3	= Clearly represented
		4	= Clearly represented and
		accurate
		5	= Clearly represented, accurate &
		current
B23	Presentation of	1 = Not structured		1	2	3	4	5
content is structured/ organised.		2 = Semi-structured
		3 = Structured
		4	= Well Structured
		5	= Well Structured & provide site
		map
B24 The content does not bias.		1 = Content is very bias		1 2 3 4 5
(Gander, race, religion, culture,		2 = Content has some element of
		biasness
localisation, etc.)		3	= Content is acceptable
		4	= Content is not bias

		5	= Content is not bias & widely
		accepted
			IMPLEMENTATION
SYSTEM REQUIREMENT
B25	Accessibility of the	1 = Most of the content are not			1	2	3	4	5
	content.	accessible (e.g. broken links, poor
		response time = > 5min. to load a
		page)
		2	= some of the content are
		accessible
		3	= Most of the content is
		accessible
		4	= All content are accessible
		5	= All content are accessible with
		good response time (<10s)
USAGE AND SUPPORT
B26	Clear instructions are	1 = No instructions provided (i.e.			1	2	3	4	5
	available on how to	very difficult to use)
use the content.		2 = Some instructions provided
		3	= Most instructions provided
		4	= All necessary inst. provided
		5	= All necessary inst. provided
		with additional help tools (i.e. very
		easy to use)
B27	The support is	1 = No support is provided			1	2	3	4	5
available.		2 = Some support is provided (e.g.
		via e-mail)
		3	= Support is available with good
		response time (e.g. feedback within
		a day)
		4	= Immediate support is available
		during office hours
		5	= 24 x7 immediate support is
		available
B28	It provides support	1 = No support			1	2	3	4	5
for special needs (i.e. handicap).		2 = Little support
		3 = Some support
		4	= Good support
		5	= Good support for diff. handicap
		people

			EVALUATION
EVALUATION
B29	Content has been	1 = No verification		1	2	3	4	5
	endorsed/verified	2 = Verified by non-SME
	by SME.	3	= Verified by SME (e.g. a
		teacher)
		4	= Verified by experienced SME
		(e.g. teacher with >5 years
		experience)
		5	= Verified by more than one
		experienced SME.
PRODUCT UPDATES
B30	Upgrades are	1 = No upgrade is available		1	2	3	4	5
	available.	2 = Upgrade available but once in two years
		3	= Yearly upgrade is available
		4	= Quarterly/biannual upgrade
		available
		5	= Auto-upgrade features
		included.