Powerful Assessment by Type - Dilemma-based assessment

 

Business

  • Video triggers on the key dilemmas faced by early career graduates gathered from the field – assessment involves saying how you would handle the dilemma presented and why your strategy is both feasible and is likely to be effective, given what has been learnt in the course.

(Dil)

  • Business capstone. Students have to ‘manage’ an international company with constant input of unexpected challenges.

Cap/Dil)

  • Students have an assignment in which they have to state and justify a considered position on a real world ethical dilemma from a business case-study which is currently in the press.

 (Dil)

Community & social work

  • In a final year community-based service project students work for 2 semesters with a community group on one of its key development priorities and have to demonstrate their ability to work constructively with the client, respond to their needs, develop a relevant and workable plan of action on an improvement area and commence the process of implementation.

 (Field/PBL/Dil)

  • Assessment of discretionary decision-making in social work.

(Dil)

  • Responding to scenarios of the most common challenges early career social workers experience. The assessment is based on what the student would do to manage the situation and how this is consistent with the good practice principles taught in the subject.

(Dil)

Dentistry

  • Dentistry: a real world case of a client – involving diagnosis and patient relations then interviewing a successful dental practitioner on how they would handle the same case and reporting back both in an assessment submission and in a class presentation.

 (Case/Dil/Int)

Education

  • Reflective practice: students work in groups of 3-4 to respond to a critical incident drawn from real world practice (an unexpected dilemma/challenge actually experienced in the classroom, on a field trip, in a school lab, or in dealing with parents). The dilemma is provided in the form of a video clip or a case file. For assessment, each student then writes up their own diagnosis of what is going on and how best to handle the situation in the light of this diagnosis and what has been learnt in the degree.

 (Dil/Reflect)

  • Actual examples of a range of different student work in math are given to trainee teachers with notes on the backgrounds of students – each trainee is to analyse what the student work is saying about how they are thinking, what the gaps in their skills/understanding are/what is the best aspect of each case and what most needs improvement. They are then to identify a strategy for addressing this ‘diagnosis’. Students come together in small groups to compare and contrast their diagnoses. They then implement their plan and evaluate the outcomes. Why is this powerful? It tests the ability to ‘read’ the student background and their performance and ‘match’ the most fitting response. It is authentic (the work is actually from real students). Students learn how to learn from each other, confirm their diagnosis and formulate a better response. It emphasises how the effective teacher needs to ‘read’ the unique situation and capabilities of each student and custom-tailor a response. Added to this are ‘killer moment’ scenarios – e.g. when a trainee says, “I’ll get one of my best students to help a weaker one.” To deepen the dialogue, she is asked, “What do you do if the parent of the bright student comes in and says, ‘Why are you using my daughter as a tutor when you should be giving her more advanced work so her chances of getting into an Ivy League University are optimized?” (Dominican University of California).

 (Field/PBL/Dil)

  • Trainee teachers have to select one special needs student and work with them on math or literacy.
    The assessment involves:
    • Developing a background profile of the student and identifying areas for development
    • Reading the formal assessment documentation for their selected student and then, from both (a) and (b) develop an individual management plan, with a justification
    • Implementing their plan and videoing themselves working with the student on it – noting the most ‘wicked’ moments and discussing what they did to handle the dilemma and to evaluate its effectiveness.
    • Working week-by-week with their chosen child and in University debrief class held each week discussing what went well and what didn’t with peers and the instructor. Particular attention is given to the ‘wicked moments’ and the strategies used.

The focus of assessment

    • to determine how effectively the trainee draws out the key lessons from the experience against the good practice guidelines discussed in class
    • to evaluate the personal, interpersonal and cognitive capabilities of the student along with their skills and knowledge in an integrated way.

Why is this powerful?

    • It checks the ability to reflect in action; the ability to ‘read and match’; and to change course if a planned strategy is not working.
    • It encourages ‘reading the emotional state not just the cognitive state of students. It checks trainees’ ability to manage themselves personally and their interpersonal capabilities when something doesn’t go according to plan.

(Dominican University of California)
(Case/PBL/Dil)

Engineering

  • Ethics embedded assessment – a real world case where the practitioner is faced with an ethical dilemma. The assessment is focused on what you would do, why and how this aligns with key ethical principles of professional practice as an Engineer discussed in class.

 (Case/Reflect/Dil)

English

The assignment requires students to identify a key dilemma currently experienced in their life circumstances and to:

  • Describe the dilemma
  • Look at the option first chosen to deal with it
  • Identify an alternative way it could have been handled
  • Trace out the different consequences that could have resulted depending on which option was selected
  • From this come up with a more optimal approach that would have given the best outcome possible.

The assessment task is introduced by showing a clip from the movie “Sliding Doors” which traces out the different consequences that ensue based on whether the heroine did or did not get on a train as the doors were sliding close. The instructor also models how to do the project by presenting her own sample from a dilemma she is experiencing.

This assignment measures ‘adaptability’, the ability to trace out the consequences of and evaluate alternative, potentially relevant courses of action and ‘responsiveness’ (Maricopa college system Arizona).

(Dil)

Entrepreneurship & invention courses

  • Social entrepreneurship program – An organisation active in the area gives teams of students a real world dilemma/problem around a social challenge – this is an example of ‘real-world’ ethical entrepreneurialism and requires each student to demonstrate that they are able to accurately ‘read and match’.

 (Dil)

Health

  • In the assessment of a community service unit in an allied health course students are presented with a critical incident. They have to describe clearly and accurately what is going on and from this derive an accurate diagnosis and plan of action justifying what they develop by using the capability framework and good practice principles discussed in class.

 (Dil)

ICT and software engineering

  • IT and chemistry: online interdisciplinary scenario-inquiry tasks for active learning in large, first year STEM chemistry courses with more than 1000 students enrolled – students from different disciplines are to work together to determine how best to handle a complex, real world issue with no ‘right’ answer – this was a UQ led initiative funded by ALTC/OLT in 2009.

 (Dil/PBL/Inv)

Medicine

  • Trigger videos based on real-life dilemmas identified by experienced medical practitioners are discussed in class in groups. Each group is to say how they would handle each case and why. Then, they compare and contrast their strategy with what the practitioner did. At the end of the semester in a final examination, they are faced with a new, unseen dilemma and have to say what they would do to handle it and why, explaining how their strategy aligns with the effective approaches and key capabilities explored during the course.

(Dil)

  • ‘Long cases’ in medicine – high stakes and developed from real life dilemmas and cases faced by early career medical practitioners.

 (RPS/Dil)

  • Scenario for intensive care medicine specialists: a tragedy has occurred and the patient is brain dead. Her spouse is in another city and is flying in and has only been told that there has been a bad accident. Students have to say how they will break the news and broach the issue of organ donation. Assessment is outcomes-based and criterion-referenced (RPS/Dil).
  • Scenario assessment: how, as a male doctor will you undertake a gender and culturally-sensitive physical examination of a female patient who is a refugee from the Middle East. Explain and justify your approach.

 (RPS/Dil)

  • The modified exam question or MEQ (Feletti, G.I. and Engel, C.E., ‘The modified essay question for testing problem-solving skills’. The Medical Journal of Australia, volume 1, number 2, Jan 1980 pp79-80) is based on a scenario in which the individual student is to take on the role of a professional early in their career working in a specified context. A practical example provided by Professor J Knox is included in the University of Glasgow’s Introduction to Assessment (McCulloch, M, L&T Centre 2007) pg 22 :

Prang

Page 1
Do NOT look through this booklet before you start. Answer briefly each of the four questions in turn completing each one before moving to the next.
.Do not go back and add to or alter what you have written
Page 2
It is your night off, and you are relaxing at home. At 22.30 you are startled by the sound of breaking glass and crumpling metal outside your house. You rush out and, in the dark, dimly discern a small shattered sports car on the pavement, wedged between the wall and a lamp post. List but do not elaborate on, the main points in your plan of action, putting what you consider to be the most important actions first.
Page 3
The car lights are still lit and in their glow you see a sole occupant trying in vain to get out through the off side door, which is jammed. Petrol is pouring out from the shattered tank. Already passing cars have stopped and people are running towards the scene of the accident. What immediate specific actions do you take, and why?
Page 4
As the driver stumbles out through the nearside door he says, “its all right, I’m a doctor”. You recognise him as one of your partner’s patients who is working as a pre-registration house physician in the local hospital, half a mile down the road. His breath smells strongly of alcohol and he says in an over deliberate way – “Course, I’m under the influence – had six pints – was going too fast – skidded, lost control – bang!” Miraculously, he appears to have escaped without any physical injury, though he is pale and shaken. What do you consider you should do next?
Page 5
In fact you run him up to the accident department of the hospital where he works and leave him with the duty surgical registrar. On your return home, half an hour later, you find the scene of the accident swarming with police, firemen and breakdown personnel. As you put your car away you wonder if you have discharged completely your responsibilities. List, but do not elaborate upon, the various factors which influence your decisions about your next actions.

(RPS/Dil)

Nursing

  • Simulations of the real world dilemmas that arise in practice identified by successful early nursing practitioners – tests the student’s ability to apply skills and knowledge to a unique situation, diagnosis, client relations and capacity to deliver and evaluate the results.

 (RPS/Dil)

  • Managing, assessing and helping a deteriorating patient using a de-identified, real world case, with increasing challenges unfolding over time. Assessment is based on the quality of analysis, diagnosis, how well the management plan matches the condition of the patient, and the quality of interpersonal skills with not only the client but also with the family.

 (Case/Dil)

  • Evidence based practice focused on handling real-world dilemmas faced during the practicum. The practicum report takes one dilemma faced and the trainee writes up what happened, how it was handled, the outcomes and compares their perceptions with the feedback from the supervisor on the effectiveness of the approach using the top 12 key capabilities identified by successful early career nurses as an evaluation framework.

 (Dil)

Occupational Therapy

In a simulation-based assessment task students are given a specific role as part of an O.T. team and confronted with a case where there are number of possible ways to respond to the case as outlined. Students have to say what way of proceeding is likely to be most productive and why. This task gives focus to seeing how well the student would handle the most common real-world dilemmas that occur in early career practice.

(RPS/Dil)