Use calibration to ensure that grading is reliable
- What is good practice in ensuring students and staff are clear on how to validly and consistently allocate different grades?
The approach to assuring the clarity, validity and reliability (consistency) of grading is ideally confirmed and supported by a process of intermarker ‘calibration’ (see references below). The process for ensuring inter-marker calibration discussed at the workshops includes attention to the steps summarised in Box Nine.
A suggested process for using inter-marker 'calibration'
A small group of experienced graders in a unit of study comes together each year:
It was noted that producing the self-teaching guide for grading suggested in Box Nine is one way of avoiding the extensive time and opportunity costs of repeatedly bringing all markers (full time and sessional) together for face-face consensus building.
It was also strongly recommended that the results be shared with students at the outset of each unit of study using exemplars of different grade levels so that students know exactly how grades will be allocated; and that the use of assessment-focused learning guides for each unit of study similar to those developed at universities like Western Sydney University be considered.
- Barrie et al (2012): Assessing and assuring Australian graduate Learning outcomes, OLT, Sydney pgs 48ff.
- Griffith University: Assessment matters! – click on the section marked ‘consensus moderation’.
- Hancock, P & Freeman, M (2014): Achievement matters: external peer review of accounting standards, OLT, Sydney. See especially Sections 3.3: External calibration and 3.4: External Assurance pgs 25ff and the discussion of project success factors (Section 6.1), Challenges (Section 6.2) and Key Lessons (Section 6.3) on pgs 49ff.