Handbook on the Evaluation of Courses and Teaching: a guide for Teaching Staff and Academic Heads 2009



Download 395,63 Kb.
Page1/4
Date conversion07.12.2017
Size395,63 Kb.
  1   2   3   4
Handbook on the Evaluation of

Courses and Teaching:
A Guide for Teaching Staff and Academic Heads
2009

http://www.auckland.ac.nz/uoa/home/about/teaching-learning/evaluation

Comments and suggestions to:
Dr. David Tippin

Quality Co-ordinator

Planning and Quality Office

d.tippin@auckland.ac.nz
Table of contents


  1. Evaluation purposes and context 3




  1. Roles and responsibilities concerning evaluation 5




  1. Resources and further assistance 6




  1. The Evaluation Plan 8




  1. Summative (end of course) evaluations 9




  1. Gathering student feedback during a course:

‘formative feedback’ 17


  1. Peer observation and review of teaching 19

Appendices


1. University statement on student feedback 21
2. Policy on Student evaluation of courses and teaching (2008) 23
3. Overview: the evaluations process 28
4. Template for a 3-year Evaluation Plan 29
5. Generic questions for end of course evaluations 30
6. Other questions (supplementary questions) 32
7. Peer observation and review

(two templates for recording comments and suggestions) 52


Handbook on the Evaluation of Courses and Teaching:
A Guide for Teaching Staff and Academic Heads
2009
1. Evaluation purposes and context
1.1 Introduction

This handbook has 3 goals:



  • To provide a brief introduction to the University’s expectations concerning evaluation of courses and teaching

  • To describe the principal evaluation methods and procedures that are encouraged and/or supported by the University

  • To outline the steps involved in the planning, commissioning and administration of evaluation forms, and the interpretation and use of evaluation results

The handbook covers three evaluation methods:



  • End-of-course evaluations using standardised forms

  • Gathering student feedback during a course

  • Peer observation and review of teaching


1.2 The purposes of evaluation by students

The ultimate purpose of evaluation of courses and teaching by students is to provide information that can be used to improve student learning. In support of this objective, evaluation can also be used to:



  • provide feedback on the quality of courses, teaching and tutoring

  • provide information on how students are experiencing a course and its teaching

  • assist staff in building a profile of their strengths in teaching and learning

  • identify areas for enhancement and improvement

  • suggest areas where further professional development should be considered.

Information generated from evaluations by students, peer review and observation of teaching, and other methods can be assembled by staff in a personal teaching portfolio.


1.3 The context of evaluation

Student feedback obtained through evaluation forms – whether gathered toward the end of a course (summative evaluation), or at the early or mid-way points of the course (formative evaluation) – is an intensively researched area in higher education internationally. The research suggests, for example, that students can be fair and reasonable judges about the quality of teaching that they experience, e.g., the extent to which academic staff are well-prepared for classes, effectiveness of communication, and the timeliness and usefulness of feedback. The research is somewhat more equivocal on how well-positioned students – especially at the undergraduate level – are to make judgments about the content and relevance of a particular course.


The issues identified in the research literature are complex. This complexity suggests that the results of evaluations by students must be analysed in context and, most importantly, not in isolation from other feedback and monitoring mechanisms. This is why this Handbook also includes a section on peer observation and review, a technique which can be used to provide an additional professional perspective on teaching and courses that is sensitive to such variables as disciplinary context and standards, and the nature and capabilities of the students. It is also recognised that courses and teaching are monitored in the regular discussions and feedback sessions that occur among contributing staff, through the use of course review forms, and during more formal processes of programme, accreditation and departmental reviews.
The University encourages Academic Heads and teaching staff to use multiple methods of evaluation to provide a fully rounded and contextualised perspective on the quality of teaching and courses and, ultimately, student learning outcomes.

2. Roles and responsibilities concerning evaluation
2.1 Academic Heads

Academic Heads have responsibility for monitoring of teaching quality and for initiating and overseeing regular reviews of the quality of the curriculum and teaching in their department, division or school. Academic Heads are responsible for ensuring that staff are supported in their career development, for conducting performance and development reviews, and for recognizing and supporting excellence in teaching. Academic Heads also oversee preparation of the annual update to the 3-year rolling Evaluation Plan.


2.2 Other oversight/monitoring bodies and positions

In some faculties, there are other bodies (e.g., Boards of Studies and equivalents) or positions (e.g., programme leaders) that may have responsibilities that pertain to the quality of courses and the curriculum. These responsibilities may require access to course evaluation results, with the agreement of the responsible Academic Head.


2.3 Teaching staff

Teaching staff use a variety of means to monitor the quality of their courses and teaching. These include evaluations by students, and also self-review, collegial peer inputs, and benchmarking. Teaching staff also monitor the quality of the teaching of staff for whom they have responsibility, including tutors and demonstrators. Teaching staff should maintain an open dialogue with students concerning the evaluation process and the purposes for which the results may be used.


2.4 Students

The teaching and learning process is a partnership between students and staff, with students being ultimately responsible for their learning. Consistent with this, students have a responsibility to provide periodic feedback to teaching staff on the learning process and its outcomes. Part of this feedback occurs through the formal evaluation process. To be effective, student feedback must be well-considered, frank and constructive.


Effective 2009, incoming undergraduate students are being provided with a statement of their roles and responsibilities to provide feedback through various means – including evaluations – concerning their University experiences. This statement appears as Appendix 1.
2.5 Teaching and Learning Quality Committee (TLQC)

TLQC, a sub-committee of Education Committee that is chaired by the Deputy Vice-Chancellor (Academic), receives and considers aggregate reports on evaluations results (prepared by the Planning and Quality Office), and a report on the processing of evaluations forms (prepared by the Evaluations and Scanning Centre). Each faculty is represented on the Committee. Faculties, schools and departments may have Teaching and Learning Committees whose responsibilities include oversight of their evaluation processes within their unit.



3. Resources and further assistance

3.1 Student Evaluation of Teaching and Courses (2008)

This University policy provides the framework for the conduct of teaching, course and tutoring evaluations. All teaching staff and Academic Heads should be aware of its contents. See Appendix 2.



3.2 Effective Teaching at The University of Auckland (2004)

This statement, available on the Teaching and Learning website, includes a description of techniques to monitor and assure the quality of teaching and learning.



http://www.auckland.ac.nz/uoa/fms/default/uoa/about/teaching/policiesprocedures/docs/effectiveteaching.pdf

3.3 Evaluations Website

http://www.auckland.ac.nz/uoa/home/about/teaching-learning/evaluation
The website is a staff resource for the evaluation of courses and teaching. It contains:

  • An e-version of this Handbook

  • The access point for the commissioning and creation of evaluation forms

http://cecil.auckland.ac.nz/esc


  • Links to the service units that can provide advice and assistance for various aspects of the evaluations process



3.4 Assistance and support

Three service units provide assistance and/or support for various stages of the evaluations process:




  1. Evaluations and Scanning Centre

Completed evaluation forms are sent for scanning and processing to the Centre. The Centre verifies that student responses are scanned accurately, and reports any discrepancies or quality issues back to the teaching staff concerned. After processing, the Centre forwards the collated results back to the teaching staff. If the evaluation is part of the Evaluation Plan for courses and teaching, the Academic Head also receives a copy of the collated results.
Turnaround time for processing and reporting can range from a few days to over a week, depending on Centre workload. Processing workload is particularly heavy at the end of each semester.
The Centre is responsible for ensuring that confidentiality and privacy of teaching staff is maintained throughout the processing and reporting processes.

Contact: evaluations@auckland.ac.nz ext. 88325




  1. Planning and Quality Office

The Office, through the Quality Co-ordinator, is responsible for maintaining and assuring the overall quality of the student evaluations process. This has a number of aspects. The Office:

  • Advises, on request, on the preparation of teaching unit Evaluation Plans, and monitors implementation of these plans. A copy of the Plan must be lodged by each teaching unit with the Office in March of each year.

  • Provides, on request, assistance with statistical interpretation of reports

  • Maintains the question item bank and responds to any request for additions

  • Maintains the student evaluations page on the teaching and learning website

  • Prepares aggregate reports for Teaching and Learning Quality Committee, and faculties

Contact: quality@auckland.ac.nz ext. 84661




  1. Centre for Academic Development (CAD)

The Academic Practice Group of CAD is the principal source of advice and assistance on any pedagogical and teaching and learning issues that may arise from student evaluation results. The Group can also provide advice to staff and Academic Heads concerning any professional development matters.
Contact: cadreception@auckland.ac.nz ext. 85132

3.5 Overview of the evaluations process

The diagram presented in Appendix 3 (‘The Evaluations Process’) provides an overview of the major features of the University evaluations process, and the roles of principal participants.




4. The Evaluation Plan
4.1 Requirements in preparing a Plan

This Plan, explained in the Student Evaluation of Teaching and Courses policy, provides the formal framework within which summative evaluations are to be conducted within each teaching unit. Its major features are:



  • A 3-year time horizon (to align with the Evaluation Policy’s minimum requirements that all undergraduate courses and teaching, and postgraduate courses of a significant size, be evaluated at least once every 3 years)

  • The Plan is prepared annually within each teaching unit

  • The Plan includes summative course and teaching evaluations. Both undergraduate and postgraduate evaluations are to be included.

  • A copy (signed by the Academic Head) is to be submitted to the relevant Faculty Office and the Planning and Quality Office by 31 March of each year. Circumstances may arise in the course of the academic year that require the Plan to be adjusted. Significant changes made to the Plan after its submission should also be forwarded to these two Offices.

  • Results of evaluations commissioned through the Plan are conveyed both to the teaching staff concerned, and their Academic Head

The Plan should be prepared with at least three considerations in mind:



  • Meeting the requirements of the Evaluation Policy in terms of coverage and frequency, both of courses and teachers

  • Assisting with the evaluation of the performance of newly appointed staff, and staff who anticipate applying for continuation, promotion, teaching awards, etc

  • Taking into account future data needs that might support a planned or anticipated departmental or established programme review, a Graduating Year Review of a new programme, or a professional accreditation exercise.


4.2 Template for the Plan

A template to assist Academic Heads in constructing a 3-year Evaluation Plan is included as Appendix 4 of this Handbook.


4.3 Course evaluations as a teaching unit resource

A good practice for summative course evaluations that are conducted as part of the Evaluation Plan is for the results of these to be held within the department as a resource for staff. Each course might have a Course Portfolio or equivalent which brings together course materials, student course evaluations, actions taken to improve the course etc. New teachers of the course could have access to these materials.


4.4 Implementation of evaluation policy

The Planning and Quality Office will audit, on a sample basis, Evaluation Plans and their implementation against policy, with attention given to three aspects:



  • Verifying that all new undergraduate courses are evaluated towards the end of their first offering

  • Verifying that established undergraduate courses and postgraduate courses of significant size are evaluated at least every 3 years

  • Ensuring more generally that students are not being requested to complete too many or too few evaluations


4.5 Evaluations conducted in addition to the Evaluation Plan

The Evaluation Policy describes a number of circumstances in which either the Academic Head or a member of academic staff may commission evaluations which are not scheduled in the Evaluation Plan. Two points should be noted:



  • Results from an evaluation not included in the Plan are conveyed only to the teaching staff and/or Academic Head who commissioned the evaluation

  • Results from these evaluations are not included in aggregate University reporting


5. Summative (end of course) evaluations
The most common types of evaluations conducted at the University are those which occur toward the end of a course. These ‘summative’ evaluations, usually conducted in-class on pre-printed forms, help provide an overall student assessment of the course and/or its teaching/tutoring.
5.1 A summative evaluation, and/or obtaining feedback during the course?

Teaching staff sometimes ask when it is appropriate to do a summative evaluation at the end of a course, as opposed to obtaining feedback during a course. The following guidelines apply:



  1. Summative evaluations are required periodically for both courses and teaching. Their frequency is set out in the Student Evaluation of Courses and Teaching policy (see Appendix 2, and the ‘Frequency of evaluations’ section below). The scheduling of these required evaluations is planned through the teaching unit’s Evaluation Plan.

  2. Staff may seek student feedback during a course in order to monitor or improve its teaching or delivery more quickly. An in-course evaluation may be done even if a summative evaluation is scheduled to be administered as the course nears its end. These ‘formative’ evaluations may be done through a short formal questionnaire, or through a class discussion/feedback session.

  3. Teaching staff may also conduct a summative evaluation for their own needs when an evaluation is not scheduled as part of the Evaluation Plan.


5.2 The types of summative evaluation forms

Six standardised forms are available for use in end of course evaluations:





Instrument

Number of

generic (compulsory)

questions


Evaluation focus










For undergraduates:







LS – Lecturing

9

Teaching practices

TS – Tutoring

10

Tutoring

CS – Course

11

Course content, design and delivery










For postgraduates:







PG – Postgraduate

10

Teaching practices, and course content, design and delivery










Other:







CL – Clinical attachment/placement

6

Clinical attachments and placements

GE – General Education

5

The General Education aspects only of courses listed on the General Education schedule. ‘GE’ evaluations are commissioned by the General Education Board of Studies.


5.3 Question content

Generic questions:

To provide a consistent approach across faculties and courses, each of the 6 evaluation forms described in the table above contain a number of standard questions. The reverse side of the evaluation form contains two open-ended questions (three in the case of the PG form) to which the students may respond in their own words. The standard questions are compulsory and are not to be deleted or have their wording modified.


The generic items for each questionnaire are displayed in Appendix 5.
Supplementary questions:

To provide greater sensitivity to the diversity of teaching methods and course delivery contexts at the University (e.g., online courses, laboratories, fieldwork, practicums and studio settings), or to allow more detailed exploration of specific issues, each of the six generic evaluation forms may be adjusted by adding a small number of supplementary questions. These additional questions are displayed in Appendix 6.


5.4 Frequency of summative evaluations

  1. New courses. There must be an assurance that a new course is working well, and is well-received by students. Therefore, all new courses should be evaluated towards the end of their first offering, using the ‘CS’ evaluation form (for undergraduate courses) or the ‘PG’ evaluation form (for postgraduate courses of a significant size). The second offering of the course may also be evaluated if deemed necessary or appropriate. For example, if the first evaluation reveals problems and then improvements are made, evaluation of the second offering could be used to help determine if the improvements have been effective.




  1. Established courses. There should be a periodic assurance that an established undergraduate or postgraduate course is satisfactory from a student perspective. These courses should be evaluated at least once every three years (assuming the course is offered in consecutive years), using the ‘CS’ and ‘PG’ evaluation forms as appropriate.

Some courses may be offered intermittently, e.g., every other year. In such cases the ‘at least once every three years’ provision should be interpreted as ‘at least every third offering’.




  1. Teaching of each staff member. End of course feedback from students concerning teaching should be sought periodically. This should occur at least once every three years. The ‘LS’ evaluation form (undergraduates) and ‘PG’ evaluation form (postgraduates) are to be used.




  1. Tutoring. As part of their responsibility for overall course quality, course convenors should arrange for periodic student evaluation of tutoring in their courses, bearing in mind the turnover of tutors each year. The ‘TS’ evaluation form is to be used.


5.5 Over-evaluation and under-evaluation

The policy settings concerning evaluation frequency are designed to strike a balance between too much and too little evaluation of courses and teaching (e.g., evaluating every course at every offering in successive years, or not evaluating a course at all). There may be compelling reasons why a course needs to be evaluated in successive offerings, e.g., to test students’ reactions to changes that have been made as a result of a previous evaluation, but this should be the exception rather than the rule.


In deciding frequency of evaluations, a student perspective should be borne in mind. If courses were evaluated at each offering, an undergraduate student taking a full 120-credit course load could be required to complete 8 course evaluations each year, plus a number of teaching and tutoring evaluations. Too frequent evaluation risks higher levels of student non-participation, affecting response rates and the validity of the evaluation results.
  1   2   3   4


The database is protected by copyright ©sckool.org 2016
send message

    Main page