Evaluation of eLearning for Effective Practice Guidebook/Class Projects/Mareena and Veronique - project page

= Veronique and Liz - Evaluation Plan =

[Liz - I suggest you start your introduction with an overview of the type of evaluation you are doing - in this case, because the online course has already been running, it is an Effectiveness evaluation. However, you can still investigate the usability of some of the features of the online course. In your descriptions, keep strictly to information relevant to the evaluation project. In the introduction this is about the use of an effectiveness evaluation in relation to this course.

In the background inform the reader about the course (without providing information which can identify organisations or persons) and why the evaluation is needed. For example, how long the course has been running, whether it has been evaluated previously, features and materials used in the online course, how students currently use it, challenges etc.

My advice is that you remove all references to SIT and employment difficulties - this is an open platform - and you may be opening yourself up to trouble.]

Introduction
Provide an overview of the intentions and design of evaluation project, and introduce major sections of the plan as well as the primary people involved in writing the plan.

Liz -This Evaluation Plan describes the background, purpose, sample, limitations, objectives, outcomes, audiences, decisions, questions, methods, samples, instrumentation, timeline, budget, [Liz - this should come last in the introduction.--Bronwynh 01:17, 26 October 2010 (UTC) ] for a ? Effectiveness ?usability evaluation to assist in quality improvement for an online paper in the BNsg programme at SIT.

[Introduce the reader to the evaluation plan - it should be a summary of what you are doing. Information about the course should go in the background section.--Bronwynh 01:17, 26 October 2010 (UTC) ]

Currently the unit is based on 3? multichoice quizzes, and 2 short answer text box areas that requires tutor to read and identify essential elements to meet correct criteria, the student resubmits up to 3 attempts, until mastery is achieved. the writer has concern that students drop out and dont complete the unit, or that they appear to take a long time to complete the online unit; not aware of time frames other than can commence the unit in february and must complete by november 30th...need to reconfirm this as documented anywhere in the unit outline as not clearly stated at my handover and the staff member handing over suddenly resigned and did not makeherself available for any questions from me. [this last bit is not really relevant to the evaluation project. --Bronwynh 01:17, 26 October 2010 (UTC) ]

Background - Veronique
[Veronique - this is a great plan - in the interests of time I have added suggestions where I believe changes need to be made. The rest is fine. Purposes, decisions, evaluation questions and methods all fit well with survey and interview questions. Have a look and it is the survey questions I am particularly interested in seeing changed to provide consistency and to get the information I believe you need to answer your evaluation questions. See what you think - you are almost good to go. This is going to be a very valuable project. You can use OP survey tool for questions - I can help you with this if unfamiliar. --Bronwynh 02:35, 25 November 2010 (UTC)]

In recent years Otago Polytechnic has focussed on "transforming from an organisation focussed prodominantly on face to face teaching to one able to deliver more of its programmes in flexible modes through blended delivery strategies, especially by distance and online learning" (Otago Polytechnic Strategic Plan, 2008-2010). In order to achieve this, Otago Polytechnic began developing systems and structures that support flexible/blended teaching practices and prioritized the reskilling of academic staff for teaching courses and developing programmes for flexible delivery. The eLearning technologies used to support learning at Otago Polytechic include a learning management system (Moodle), an online conferencing tool called Elluminate, and several applications for developing learning resources (examples include eXe for creating web-based course resources and Audacity for audio recording and voice-over of powerpoints).

Otago Polytechnic has a variety of support processes and resources available to academic staff. This includes online “How to” resources, individual support and small group workshops offered from experts within the Educational Development Centre, help desk support for technical problems, Moodle champions and mentors within each school that share their knowledge and skills with colleagues, and professional development courses, for example, papers in the Graduate Certificate in Tertiary Learning and Teaching provide an opportunity to experience eLearning and explore eLearning technologies for teaching and supporting learning.

The experience of Educational Developers at Otago Polytechnic suggests that new teaching staff arrive with varying levels of digital capability and limited experience or expertise of the specific eLearning technologies used by the institute. While there is a formal induction process for new staff, currently this does not include any discussion about flexible/blended delivery or an introduction to the eLearning technologies and support available to use the tools effectively. And although there are a variety of supports already in place, it is not known how easy it is for new staff to access these and how effective they are. Therefore the purpose of this evaluation is to determine whether the support processes and resources are meeting the needs of new academic staff, and to identify any improvements that can be made.

Background - Liz's focus

The writer is previously taught within the B Nsg programme at SIT and had to take over a shared coordination of an existing unit: Direction and Delegation; this was offered as a special topic at 600 level via the blackboard interface of SIT intranet.

[Only include information relevant to the evaluation project - this paragraph seems more relevant to your situation as the evaluator and is not relevant. You need to say why an effectiveness evaluation is needed - is it to ascertain the effectiveness of the online delivery design for the learners, or for the teachers? The background gives the reader information which helps them understand why the evaluation is necessary. Have there been challenges with the present design, or is it needed because the teacher is uncertain about how the students' learning is benefiting from the online design etc? --Bronwynh 01:17, 26 October 2010 (UTC)]

This unit is completed earlier in 2010 and the writer negotiated to continue a 10 week period [to march 31] when this paper re ran, to facilitate an effective hand over and evaluation to the subsequent paper coordinator. Unfortunately, the writer moved overseas over the summer and altho continued with finalising this work, was unable to complete a timely evaluation to be of use to the rerunning of this unit. This project will continue in the interests of evaluatiing effectiveness of this style of delivery as the writer is currently continuing teaching using a blackboard framework at curtin University in Perth, WA and is currently planning a new unit of teaching where blended learning is the optinmal delivery for the 2011 commencement.

Purposes
Describe the purposes of the evaluation, that is, what you are evaluating and the intended outcomes.

Liz's focus

[Re-written - I hope yo can understand it - just remove my signature and incorporate the words in square brackets. --Bronwynh 03:47, 26 October 2010 (UTC)]

The purpose of this evaluation is to assess the effectiveness [of an online course accessed by Registered Nurses for continuing education.--Bronwynh 01:59, 26 October 2010 (UTC)] [The evaluation will investigate three aspects: 1. the usability of the online materials --Bronwynh 01:59, 26 October 2010 (UTC)]; ease of on-line access; and the learning experience.--Bronwynh 01:59, 26 October 2010 (UTC)]

[the next sentence should go in the background section.--Bronwynh 01:59, 26 October 2010 (UTC) ]..specifically focussing on the Registered Nurses who have NZ Nsg council requirements to maintain professional practice ‘continuing education’ hours each 3 years re their measured competency to practice.

The intended outcomes would include an identification and validation of what contributes to effectiveness or usability of the [continuing education --Bronwynh 01:59, 26 October 2010 (UTC)] package.

Purpose - Veronique

The purpose of this evaluation is to assess the effectiveness of the processes and resources that support new academic staff in using Otago Polytechnic’s eLearning technologies for blended and distance delivery.

The evaluation will investigate how new staff use the existing resources and support systems, and whether this adequately meets their needs. It will also identify any improvements that can be made.

The outcomes from this evaluation could include:


 * a more formal organisational process for new staff to learn how to use the technologies, eg induction process
 * more accessible and effective support resources
 * changes to the way the resources and support are presented and disseminated to new staff

Limitations
''Outline any limitations to the interpretation and generalizability of the evaluation. Also describe potential threats to the reliability and validity of the evaluation design and instrumentation.''. STILL TO REVIEW AND NEED FEEDBACK RE THIS.

Liz's issues
[I would keep it general and not describe the specifics in this section. For example, unfamiliarity of the evaluator with the course means that there is little background information on which to base the evaluation. Therefore, the effectiveness of the online materials, and aspects of the course which were working well and any challenges are not clearly understood. Mention there is a time lag since some students finished the course which may impact on responses to the survey. --Bronwynh 03:45, 26 October 2010 (UTC)]

There were personal limitations for the writer having taken over the coordination of this paper part way thru the year, giving little quality time to ‘get her head around’ the course material and how the assessments worked. Also, hand over from the pervious coordinator left gaps in understandings of marking requirements as parts of this paper are ? automatically marked multi choice question’s. The writer, as stated, negotiated to continue an overrun of the unit to accomodate thos students who hadnt been adequately folowed up, or had their assessments marked; an email to each outstanding student was sent to request if they were happy to respond to an evaluation survey once they completed the unit by end March 2011. There is concern now that these students have completed this unit six months ago and how this will eaffect the response rate or the quality of the survey answers.

Limitations - Veronique

Limitations include


 * the short timeline to complete the evaluation so that the planned follow-up face-to-face or phone interview with new staff after they have completed an online questionnaire may not be included in this project
 * the small number of staff and managers that will be sampled. It is hoped that about 10 new staff who started at Otago Polytechnic this year (including both on-campus and off-campus representatives) and about 5 programme managers will participate.  With such small numbers this could be envisaged as a pilot study.

Audiences
Specify all the primary and secondary audiences or consumers of the evaluation. Liz: Sharing with local staff and for own future development of blended learning resources using blackboard in the new nursing curriculum at Curtin uni for 2011, in consideration of on line assessment. Collation/ analysis of the data will provide a report on usability and access to online materials. A report will be available for use by SIT staff at Faculty of Health, Humanities and Computing, to consider any changes to the paper design and delivery, therefore the writer will forward this survey to SIT nsg staff to re evaluate what they did with the unit this year with their current cohort of students- November, 2010

Audiences - Veronique

The people who will be interested in the outcome or could be affected by this evaluation include new staff, staff/educational developers and trainers, school heads/programme managers, and HR.

Decisions
''This section is probably the most difficult, but it should be included if the evaluation is to have meaningful impact on decision-making. Both positive and negative outcomes should be anticipated.'' YET TO BE REVIEWED

Decisions - Veronique

This evaluation could lead to [decisions about --Bronwynh 00:44, 25 November 2010 (UTC)]:


 * a more formal organisational process for new staff to learn how to use the technologies, eg induction process
 * more accessible and effective support resources
 * changes to the way the resources and support are presented and disseminated to new staff

[Note to Bron, I've already mentioned this in the Purpose section, should it be included up there or do I need to think about the Purpose section again? Veronique - what you could do is just keep this list in the Purposes section and add in the words ...decisions about..--Bronwynh 00:44, 25 November 2010 (UTC)]

Questions
''List your 'big picture' evaluation questions and sub-questions here. A key element of a sound evaluation plan is careful specification of the questions to be addressed by the evaluation design and data collection methods. The clearer and more detailed these questions are, the more likely that you will be able to provide reliable and valid answers to them. Note: The eLearning Guidelines can assist here.''

Liz's Q's
[...VERONIQUE, TO CONFIRM IF WE ARE ON THE SAME PAGE AS YET - capital letters constitute "shouting" in the online environment. :) --Bronwynh 02:24, 26 October 2010 (UTC)]

The evaluation questions are based on the eLearning Guidelines for New Zealand (http://elg.massey.ac.nz):

1. usability TD5: has a representative sample of students [change wording to - evaluated --Bronwynh 02:24, 26 October 2010 (UTC)] tested the eLearning materials and any necessary modifications been made?

Sub-questions:


 * How effective are the online materials for learning?
 * What are the factors which influence students' access to the online materials?

[this wording will enable you to consider digital skills, and attitudes to online learning as well as the design of the materials.--Bronwynh 02:24, 26 October 2010 (UTC) ]

The writer has also added other issues to explore based on discussions with Bronwyn:

2. Access to files ..were they easy to find [navigation] and download ? [this is a specific question, e.g., for the survey - you need to keep them more general for the "big picture" questions. The same applies to question number 3. --Bronwynh 02:24, 26 October 2010 (UTC) ]

3. is the interface user friendly?[time]

4. [What perceptions do Registered Nurses have about using online learning methods for continuing education? - This is a better "big picture" question, and will capture what you have mentioned. I have added some sub-questions. --Bronwynh 02:24, 26 October 2010 (UTC)]

Sub-questions:


 * What delivery methods do RNs prefer and why?

I suggest you use just these two questions and sub-questions - if they capture what you are wanting to find out:


 * TD5: has a representative sample of students evaluated the eLearning materials and any necessary modifications been made?

Sub-questions:


 * How effective are the online materials for learning?
 * What are the factors which influence students' access to the online materials?


 * What perceptions do Registered Nurses have about using online learning methods for continuing education?

Sub-question:


 * What delivery methods do RNs prefer and why?

When you decide on your sampling methods,e.g., survey, fit the survey questions under each sub-question to make sure you are asking the right things to answer them in the evaluation. --Bronwynh 02:29, 26 October 2010 (UTC)

.... for Rn’s, is this a better learning option [re access] rather than face to face teachings, inconsideration of shift work? Or does the learner have a perception that they ‘miss out’ on the social aspect and networking options available through ‘face to face‘ learning ? ..could blended learning be better in this instance?

Questions - Veronique
'''Question 1: What information about eLearning at Otago Polytechnic and the eLearning support systems and resources is required for new staff? (based on eLG TO7)'''

Sub-questions:


 * 1) What information is currently provided or available to new staff, eg at Induction, within schools, centralised (EDC, ISS)?
 * 2) Can new staff easily access the existing support systems and resources?
 * 3) What recommendations do experts suggest for providing eLearning support and information to new staff?

'''Question 2: What level of capability do new staff members need for using the organisational eLearning technologies? '''

Sub-questions:


 * 1) What knowledge and experience of eLearning and technology tools do new staff members have when they start?
 * 2) How well do the existing support systems and resources enable new staff to effectively use the eLearning technologies?
 * 3) How could the support systems and resources be improved to better meet the needs of new staff (and also existing staff)? [I would leave existing staff. You can add a note in your plan that this is an area you may explore in the future, but for now the audience to be sampled (new staff) is limited by time constraints.--Bronwynh 00:47, 25 November 2010 (UTC) ]

Methods
''Describe the evaluation design and procedures. Match the method to the purposes and questions of your evaluation, and the phase of eLearning, e.g., analysis, design, development, implementation.'' Liz's approach Yet to be decided but I will take into consideration eval tools used by others in this elearning paper, eg Tom Reeves Educational paradigm? FEEDBACK PLEASE, OR DIRECT ME TO SPECIFIC READINGS SO I CAN CONSIDER A FEW OPTIONS

[don't worry about the paradigm - I did not include it this year. Just focus on the mixed methods approach unless you are just using a survey then it will be a survey method. Whether the evaluation design is quantitative or qualitative depends on the type of questions you are going to ask. I will email you a chapter from the textbook about effectiveness evaluation.

This article may also be useful as it illustrates how to do an evaluation of the effectiveness of an online environment using an inquiry model. See: Stacey, E. &amp; Rice, M. (2002). Evaluating an online learning environment. Journal of Educational Technology, 18(3), 323-340. At: http://www.ascilite.org.au/ajet/ajet18/stacey.html --Bronwynh 02:49, 26 October 2010 (UTC)]

Methods and sampling - Veronique
[this is good. Also include some information about the concepts of summative evaluation and multiple or mixed methods as an evaluation design. --Bronwynh 00:52, 25 November 2010 (UTC)]

A summative evaluation will be carried out to determine if the support systems and resources already in place are meeting the needs of new staff. The evaluation will take a "multiple methods" approach by using a range of different methods of sampling. This is known as triangulation and strengthens the effectiveness of the evaluation by gathering information from many different perspectives. The evaluation will seek feedback from approximately 10 new staff about their experiences, and the perspective of approximately 5 school programme managers. In addition the project researcher will gain expert opinion from education/staff developers and the literature about good practice in supporting new staff. The sampling tools will include an online survey, interview and discussion forum.

Sample
''Specify exactly which students and personnel will participate in the evaluation. If necessary, a rationale for sample sizes should also be included.'' Both pre and post registration nurses will be identified participants in this study, but the writers main focus is ease of accessibility to RN’s who predominantly work shifts over a 24 hour period. Sample size will probably be less than 30 and may only get 20 or less responses due to lag since completing this unit online.

Instrumentation
''Outline all the evaluation instruments and tools to be used in the evaluation. Actual instruments, e.g., questionnaire, interview questions etc., should be included in appendices.''

Liz's tool
An EFFECTIVENESS / usability questionnaire (am currently drafting a Survey Monkey tool) will be implemented ?using a Likert Scale, and additional open ended questions ?? to allow for comments. Collation of these is for the purpose of Identification of themes, that may facilitate the development of a 2nd survey that may be used for another group currently using blackboard blended learning.

[Yes - feedback sent on email. a mix of Likert scale and open questions will give you a mix of descriptive (likert scale) and qualitative data to work with. See: Processes of Evaluation - Analysis --Bronwynh 03:15, 26 October 2010 (UTC)]

Instrumentation - Veronique
The instruments that will be used include:


 * An online questionnaire and follow-up interview for new staff to gain information on their previous experiences with eLearning technologies and determine if the current support systems and resources are meeting their needs.  Due to time constraints the follow-up interview may not be feasible for this project.
 * An interview will be conducted with selected programme managers (or heads of schools) to find out the processes in their school for introducing new staff to eLearning and eLearning technologies, and whether they feel new staff are adequately prepared and supported to use OPs eLearning technologies effectively.
 * A small focus group made up of education and staff developers will discuss the current support structures and resources in relation to staff feedback and good practice guidelines, and make recommendations for improvements.

Data Collection Matrix

Logistics and Timeline
Outline the steps of the evaluation in order of implementation, analysis, and reporting of the evaluation, and include a timeline. To meet objectives for completing this evaluation, the following will be undertaken:  Create an Evaluation plan  Implement the evaluation by collecting data: using online tool Survey Monkey  Analyse data  Complete the evaluation report Liz's elongated plan!!! March 2010 Complete marking online of unit assessments giving feedback comments to short answer questions until mastery achieved.

July reenrol in evaluation eLearning paper Oct wk 2 develop survey questionnaire Oct wk 2 end send link to Survey Monkey survey to students who have given permission to be involved, [some already agreed by phone and email to me to be particiapants]. Oct wk 3 commence colation/ analysis Oct wk 4 Data analysis completion and draft report Nov 4th Draft evaluation report Nov 14th Approval of evaluation report Dec 1st present at web conference Dec 6th final report submitted

think i need to reenter this data in a table so the formating looks much better but just in a rush right now!!

Logistics and timeline - Veronique

Budget
''Provide a rough costing of the evaluation process. For example, the evaluator's time and costs, payment of other participants etc.,''

NEED HELP TO SORT THIS ABIT, WILL GO AWAY AND CONSIDER TIME SPENT ON AVERAGE TO DATE. [Liz just look at what the others have done on their plans. --Bronwynh 03:38, 26 October 2010 (UTC)]

Budget - Veronique

Estimated time and budget: Approximately 45 hours at $50 per hour = $2250

Instruments - Veronique
Details of the sampling tools are provided below:

1. New staff members Questionnaire - online

2. Interview questions for Programme managers

3. Focus group discussion Questions for education and staff developers from EDC (Educational Development Centre at Otago Polytechnic)

New staff members Questionnaire

This questionnaire is being used to examine your experience with eLearning technologies and to determine if the current support systems and resources have met your needs as a new academic staff member at Otago Polytechnic. Your responses will be used as part of an Effectiveness Evaluation project that is being undertaken by the project researcher for professional development. The responses to this questionnaire will be confidential to the researcher and no information that could potentially identify you will be disclosed. Thank you for taking the time to complete this questionnaire. [I wonder do you need to add definitions of elearning and blended learning to this blurb so everyone is on the "same page"?--Bronwynh 01:06, 25 November 2010 (UTC)]

Section A: Technology and eLearning experience prior to starting at Otago Polytechnic 1. Which [tools and online technologies] forms of technology had you used or accessed before you started at Otago Polytechnic? Tick as many as appropriate.

Email Discussion forums/boards YouTube Facebook / My Space / Bebo Trade Me Flickr Twitter Skype Podcasts Blackboard Moodle Elluminate [Wikieducator Google documents Online games eg World of Warcraft, Sims online Virtual worlds eg Second life

[Wikipedia]

[Blogs]

[do you want to know about online banking, online news, online journals? - I would esp the last one--Bronwynh 02:29, 25 November 2010 (UTC)]

Other:

[2. Indicate which of these tools and online technologies you also use in the course of your teaching. --Bronwynh 01:06, 25 November 2010 (UTC) this will not elliminate the need for the separate question further on but remove anything which is a repeat later on. If asking this question, you can gauge if they are continuing to use technologies in their work which they were already familiar with using. --Bronwynh 01:06, 25 November 2010 (UTC)]

2. Before you started at Otago Polytechnic how confident [overall] were you in the use of computer and internet-based communication and information methods?

Extremely unconfident, Unconfident, Confident, extremely confident

3. Had you experienced blended delivery and eLearning as a student or teacher prior to commencing work at Otago Polytechnic? If yes, please describe:

[I am thinking you should also ask what software and online tools and resources they are required to use for their work.--Bronwynh 02:29, 25 November 2010 (UTC) ]

Section B: Use of support systems and resources  1. Are you an on-campus or off-campus staff member? On-campus / Off-campus

2. Were you required to use any of the following eLearning tools during your first 6 months at Otago Polytechnic? Tick as many as appropriate.

[I am not convinced that the lists should be different - perhaps keep them the same, and just ask different things about the list. Do you want to know about their confidence in using the tools? - I would as it will help to gauge the support they need for different tools. --Bronwynh 02:29, 25 November 2010 (UTC)]

Moodle Elluminate eXe Audacity Wikieducator

[Youtube]

[Flickr]

[Blogs]

[also include Audacity and exe in first question - gives a foundation. --Bronwynh 02:29, 25 November 2010 (UTC)]

Other:

3. Referring to the tools above, which support people and resources did you utilise? Tick as many as appropriate.[should you use the word Department instead of School as this will include service areas then? --Bronwynh 01:06, 25 November 2010 (UTC) ]

Support People: Colleagues in your School Colleagues from other schools Moodle champion in your school ISS help desk EDC staff Other:

Online resources: Staff Moodle Resources for ‘How to’ guides and information about Moodle Moodle doc’s Wikieducator resources EDC blog Other:

Workshops: Moodle workshops offered by EDC Digital skills workshops offered by EDC School Staff Development Workshops/Sessions Other:

4. Were you aware of all the support systems and resources [available] listed above ? Yes / No Comment:

[5. If Yes - How were you made aware of the support systems and resources?--Bronwynh 02:29, 25 November 2010 (UTC) ]

5. Did you feel you were adequately prepared and supported to use Otago Polytechnics eLearning technologies effectively? Yes / No Comment :[Please explain: --Bronwynh 02:29, 25 November 2010 (UTC)]

6. How confident do you feel you are in using these eLearning technologies now?

[since you have asked this here, you also need to know how confident they were in each tool and technology before starting at OP. If you think it is necessary to just ask about specific tools, - in light of the induction resources already provided - then do this as well and also ask if their "confidence in using the required eLearning tools during the first 6 months at Otago Polytechnic has changed AND

Also ask - How  confident overall are you in the use of computer and internet-based communication and information methods you are required to use for work? --Bronwynh 02:29, 25 November 2010 (UTC)] Extremely unconfident, Unconfident, Confident, extremely confident Comment: 7. Can you suggest any improvements in our support systems and resources? Eg would an Induction workshop on flexible learning be useful? Or an Induction workshop on Moodle?[Or a digital skills workshop?--Bronwynh 01:06, 25 November 2010 (UTC)]

Interview questions for Programme managers:

Brief introduction to Effectiveness Evaluation project: The experience of Educational Developers at Otago Polytechnic suggests that new teaching staff arrive with varying levels of digital capability and limited experience or expertise of the specific eLearning technologies used by the institute (eg Moodle, Elluminate, eXe, Audacity). The purpose of this evaluation is to determine whether the support processes and resources are meeting the needs of new academic staff, and to identify any improvements that can be made. 1. In your school do you have a process for introducing new staff to eLearning and eLearning [tools and ] technologies?

[Be consistent in use of terms - either use eLearning or computer and internet-based communication and information methods throughout - some of the tools are not really eLearning but are Web 2.0 so the long winded term might be better. --Bronwynh 02:29, 25 November 2010 (UTC)]

Prompts for discussion:


 * Is this a formal/informal process;
 * do you provide information or involve new staff in discussion about blended/distance learning at Otago Polytechnic[be consistent with terms - maybe mention computer and internet-based communication and information methods OR eLearning tools and technologies if this is the term you settle on. --Bronwynh 02:29, 25 November 2010 (UTC)]
 * do you provide information or involve new staff in discussion about your schools approach to blended/distance learning
 * is guidance provided on where and how to access information and support
 * what is the role of school mentors, buddies, technology champs, Moodle champs in supporting new staff

2. Do you feel that new staff are adequately prepared and supported to use OPs eLearning technologies effectively? If not, then how could this be improved?

Focus group discussion Questions for EDC (small group made up of education and staff developers)

There are a range of support systems and resources already in place for supporting new staff to use eLearning technologies. [use consistent terminology and define for each group you survey or interview - different terms and different understandings of terms mean you will get different responses. --Bronwynh 02:29, 25 November 2010 (UTC)] Do these meet good practice guidelines/recommendations? Are you aware from your experiences with staff of any difficulties / problems / limitations with the support systems and resources? What improvements could be made?