Type of Powerful Assessment - Role-play/simulation

Archaelogy

 (RPS)

Arts & Social Sciences

  • Participate in a simulated interview with peer and lecturer review against the effective interview checkpoints discussed in class with illustrative videos. Then use the feedback to write an evaluative essay on the outcomes drawing appropriately on the key content covered in the course.

 (RPS)

  • Students have to develop a proposal to improve an existing policy addressing a key social issue including strategies and the indicators they would use to measure its effectiveness (issues can include treatment of asylum seekers, aboriginal incarceration, disability rights, domestic violence; bullying in schools; the Ice epidemic) and justify the changes advocated. They have to come up with a catchy policy name, present their proposal in a simulation where they are applying for seed funding to test the proposal in practice. They have to include a justification of their approach by drawing upon all that has been learnt in the course.

(PBL/RPS)

Business

  • Production for a simulated company of budgets with a clear rationale and justification.

 (RPS)

  • A business simulation game in which teams of 3 each play a real world (and carefully briefed) role that involves them in first developing a ‘pitch’ to a specific type of investor and then ‘fronting’ people from industry. Assessment is based on evaluation from industry, the extent to which what has been learnt is effectively applied to the pitch and a self-evaluation of the effectiveness of the group process against key checkpoints discussed in advance.

 (Cap/RPS)

  • Each student undertakes a speed interview for a job – the equivalent of speed dating with assessment based on a comparison of how well the student thought they went with how well the ‘interviewer’ thought they performed against a set of ‘effective’ interview principles discussed in class.

 (RPS)

  • Real-world project using a Wiki tool and a simulation
    An online collaborative writing activity using a Wiki tool on the course Blackboard site is a part of the course project that expands over a period of 10 weeks with diverse types of activities. Students in groups create an imaginary company, collaboratively write their company profile and a job offer, build a company website (Blog tool on Blackboard site) and prepare a group presentation. Playing the role of a hiring committee, groups present their company profile and a job offer to invite candidates-classmates to apply to their company. Students-candidates prepare their resume and cover letter according to the company’s information. The company evaluates applications, selects a candidate, and produces a report justifying their selection in terms of specified criteria. The project aims to assess, in a progressive manner, students’ written productions and analytical reading in business French, oral communication skills, ability to evaluate their own learning (through reflection writings), as well as their peers’ production (University of Toronto).

(RPS/ICT)

Creative industries, Arts & Design

  • In a digital media course students undertake warcraft online and produce a video on the experience with a critical commentary on its design and impact (World of Warcraft is a multiplayer online role-playing game created in 2004 by Blizzard Entertainment. It is the fourth released game set in the fantasy Warcraft universe, which was first introduced by Warcraft: Orcs & Humans in 1994).

 (RPS/ICT)

Dentistry

  • Dentistry – working with virtual patients – using inquiry based learning and assessment. The University is working in partnership with dental schools at other universities within and beyond the nation on this.

 (RPS)

  • A communication skills training program was developed for 2nd year dental students at the Faculty of Dentistry. It includes didactic content, modeling behaviour, and practical sessions with students working in small groups using Standardized Patients and role-playing in eight different communication scenarios commonly found in clinical practice. Communication is assessed using the Kalamazoo Essential Elements Communication Checklist, which is a validated scale that measures 7 elements of communication on a scale of 1 (poor) to 5 (excellent). Four assessment strategies are used: (1) Each student/dentist’s communication is video-taped for subsequent review. (2) The student/dentist self-evaluates his or her communication skills, in addition to receiving written KEECC assessment provided by the group facilitator, patient, and observing fellow students. (3) Verbal feedback is provided by the group to the student/dentist, and (4) each student/dentist writes a reflection of their experience as the dentist. Criteria - The overall intent is for students to reflect on their own communication skills, to try and put themselves in the shoes of the patient to understand their patient’s perspective, and to provide constructive comments on observed communication skills of their peers (University of Toronto).

 (RPS)

Education

  • Micro teaching with peer and supervisor assessment against an effective teaching scale discussed in class prior to the assessment activity.

 (RPS)

Engineering

  • In the concrete structures unit of an Engineering program students design an RC concrete beam against a given brief/set of conditions. They physically prepare the beam in a team & test it to breaking point and then analyse the data, including making direct links to the relevant theory and research. Students again in a team run tests on a beam that is a critical construction element. This assessment task requires students to bring together technical skills, diagnosis and the ability to work reciprocally and constructively as part of a design and construction team.

(PBL/RPS)

  • An engineering simulation – you have to apply for a specific job so, given the criteria in the advertisement provided to you, what is your pitch and why?

 (RPS)

  • Engineering is being integrated with the liberal arts in a range of US universities.This enables students to learn not only about engineering-design principles and their technical application but also about the social context in which these designs must be put into action. This article argues that engineering education (and its assessment) today needs to help students devise innovative solutions for a complex world while also anticipating their potential unintended consequences. In other words, engineering education needs to prepare students to design expansively and imaginatively. One example cited in the article concerns the civil-engineering course at Worcester Polytechnic Institute. This course transports students to the late 19th century and challenges them to decide how best to clean up Worcester’s heavily polluted Blackstone River using the technologies available at the time. Students are confronted with a range of possible approaches — "the cheapest approach, the approach that will last the longest, the approach most likely to make Worcester an exemplar of advanced engineering design, the approach that will be least disruptive to businesses, or the approach most likely to ensure the just treatment of all residents of Worcester and along the Blackstone River" — against a historical backdrop of African-American migration from the rural South to the industrial North and concerns about threats to public health from the growth in manufacturing. At:http://chronicle.com/article/Bringing-the-Liberal-Arts to/229671/?cid=wb&utm_source=wb&utm_medium=en

(RPS)

  • A curriculum review simulation involves students in a co-creation project in chemical engineering and applied chemistry. For this summative assessment a project has been developed that encourages students to analyze the existing curriculum and then recommend and provide a rationale for changes. The project requires students to imagine that they represent the Chemical Engineering Department as they present a pitch to a group of interested donors. They must present a plan that earmarks the donor funds for a particular course-based or program level improvement. Students are directed to: “review the key learning outcomes of the 2nd year curriculum to consider how your courses work together to develop your skills and knowledge, and to identify gaps or areas for improvement in this curriculum.” In their argument students can draw on a range of sources including research and popular papers, while also integrating their firsthand experience as students.
  • The assessment criteria include clear and persuasive messages through effective argumentation; analyzing audience and purpose to select the most effective mode/genre of communication, and level of detail required; finding and properly integrating relevant information to support their purpose and argument; summarizing and synthesizing information from external sources; effectively organizing information and prioritizing it in each mode of communication (written, visual, oral) to convey a core message; applying effective strategies to the design of text, visuals and oral presentations; developing self-confidence in your process of communication (University of Toronto).

 (RPS/Co-C)

Entrepreneurship & invention programs

  • Technology and entrepreneurship @ SFU. Over 2 years engineering and business students work together. Funded by BC Innovation Council and Simon Fraser University. Business students undertake a beginners engineering course and the engineers do a beginning business course. Assessment involves developing a ‘pitch’ for a $25K prototype grant in a dragons den like competition. Assessment criteria focus on the quality of each individual’s contribution to the team, the quality of their ‘elevator pitch’, video and their final presentation along with the quality of individual reflection.

(Simon Fraser University)
(Inv/RPS)

Global Affairs

Simulation in Global Affairs (transdisciplinary)

An interdisciplinary course for First-Year undergraduates called “Order and Disorder: Global Affairs and Emerging Technologies” (WDW 152) draws upon Political Science, Sociology, History, Economics, and Philosophy.

Students are randomly divided into two groups – Soviet leaders and American leaders. The groups move to separate rooms and each receives initial instructions informing them that the year is 1983 and describing the tense state of superpower relations. Each group then receives a series of “intelligence reports” suggesting that the adversary has launched a nuclear attack. The groups must then reason through their response, discussing what they already know, what they would like to know, and what their range of options might be. Throughout the discussion a stopwatch ticks down the time until the nuclear missiles reach their target. When the stopwatch rings the groups reconvene and each side announces its response. The exercise ends either in a nuclear war or peace.

The decision making processes on both sides are then compared with a short video about a real-life nuclear crisis that resembles the simulation. This is followed by students writing up, comparing and contrasting their decision making processes and outcomes with those of the real-world political leaders. For more information: http://wdw.utoronto.ca/wdwone/ (University of Toronto)

(RPS)

Health

  • In a Public Health program students have to write a ministerial briefing on an issue that is based on a real world case – this requires a clear understanding of evidence-based practice; what motivates a minister and how the political process works and how lobbying groups operate.

 (RPS/Case)

  • In a communication skills subject in Health Science students watch a video of an interaction with a client and have to take notes exactly just as they will have to do in a clinic after they graduate. They then have to write up a progress note which fits the legal requirements and conventions and which is relevant to what is seen and is coherent.

 (RPS/PBL)

ICT and software engineering

  • Identify a project from a given menu or one of your own choosing, form your own group of fellow students with the complementary skills necessary to deliver the develop, jointly formulate a project proposal and present this to a panel – assessment is based on both the quality of the outcome and the group process.

 (RPS/PBL)

Languages

  • In an English as asecond language transition programs students from non-English speaking backgrounds practice scenarios of how they will participate constructively in seminars when and if they get a place at university. They are coached by successful senior university students from their background who receive a community service subject credit for their work with the student.

 (RPS)

Law

(RPS)

  • International law: students role-play a country in an international dispute. The assessment is against a set of clearly identified capability tests and the extent to which students can successfully take into account cultural differences.

 (RPS)

  • Students are to write a letter of advice to a client using a range of data provided in the course with a commentary explaining the strategy adopted in the letter and its focus. This tests diagnosis; the ability to link the facts and identify what might be the best way to respond.

 (RPS)

Marketing

  • One of our partner businesses presents students with a real problem. Students have to identify a relevant and feasible solution that will be cost-beneficial and, at the same time, figure out how best to ‘sell’ it to the client. This requires lateral thinking, understanding what motivates the client, ability to influence, clear, sharp and engaging presentation skills, an ability to think on your feet when the client asks a curly question during the presentation etc. Student teams present to the clients in a format similar to Dragons’ Den.

 (PBL/RPS)

  • Marketing: a simulation – you are person ‘X’ in a marketing firm (full profile details for the person are given) and you have to pitch a given real world marketing idea to industry – with a review from a ‘panel’ of classmates led by the lecturer. The student then writes up a reflective report based on the feedback, the presentation principles discussed in class and identifies what they would do again or differently.

 (RPS)

Medical Imaging

  • In a role play the student is given a patient request including a clinical history and has to diagnose what needs to be done and why. The assessor asks the student to interpret the form and nominate projections to be taken, given that specific history. The ‘patient’ (a volunteer) is waiting outside the room and the student has to discuss and justify the selected treatment with the ‘patient’. This tests the ability to ‘read and match’, clinical and technical knowledge and personal/interpersonal capabilities. The role play is evaluated by the supervisor using an agreed rubric and takes into account the feedback from ‘the patient’.

 (RPS)

Medicine

  • ‘The stations’ approach in Medicine including assessment of ‘emotional intelligence’ on entry
    • First done as a formative task – assessment for learning
    • Then at the end of the year it is done as a summative task – assessment of learning.

 (RPS)

  • ‘Long cases’ in medicine – high stakes and developed from real life dilemmas and cases faced by early career medical practitioners.

 (RPS/Dil)

  • Scenario for intensive care medicine specialists: a tragedy has occurred and the patient is brain dead. Her spouse is in another city and is flying in and has only been told that there has been a bad accident. Students have to say how they will break the news and broach the issue of organ donation. Assessment is outcomes-based and criterion-referenced.

 (RPS/Dil)

  • Scenario assessment: how, as a male doctor will you undertake a gender and culturally-sensitive physical examination of a female patient who is a refugee from the Middle East. Explain and justify your approach.

 (RPS/Dil)

  • The modified exam question or MEQ (Feletti, G.I. and Engel, C.E., ‘The modified essay question for testing problem-solving skills’. The Medical Journal of Australia, volume 1, number 2, Jan 1980 pp79-80) is based on a scenario in which the individual student is to take on the role of a professional early in their career working in a specified context. A practical example provided by Professor J Knox is included in the University of Glasgow’s Introduction to Assessment (McCulloch, M, L&T Centre 2007) pg 22 :

Prang

 

Page 1
Do NOT look through this booklet before you start. Answer briefly each of the four questions in turn completing each one before moving to the next.
.Do not go back and add to or alter what you have written
Page 2
It is your night off, and you are relaxing at home. At 22.30 you are startled by the sound of breaking glass and crumpling metal outside your house. You rush out and, in the dark, dimly discern a small shattered sports car on the pavement, wedged between the wall and a lamp post. List but do not elaborate on, the main points in your plan of action, putting what you consider to be the most important actions first.
Page 3
The car lights are still lit and in their glow you see a sole occupant trying in vain to get out through the off side door, which is jammed. Petrol is pouring out from the shattered tank. Already passing cars have stopped and people are running towards the scene of the accident. What immediate specific actions do you take, and why?
Page 4
As the driver stumbles out through the nearside door he says, “its all right, I’m a doctor”. You recognise him as one of your partner’s patients who is working as a pre-registration house physician in the local hospital, half a mile down the road. His breath smells strongly of alcohol and he says in an over deliberate way – “Course, I’m under the influence – had six pints – was going too fast – skidded, lost control – bang!” Miraculously, he appears to have escaped without any physical injury, though he is pale and shaken. What do you consider you should do next?
Page 5
In fact you run him up to the accident department of the hospital where he works and leave him with the duty surgical registrar. On your return home, half an hour later, you find the scene of the accident swarming with police, firemen and breakdown personnel. As you put your car away you wonder if you have discharged completely your responsibilities. List, but do not elaborate upon, the various factors which influence your decisions about your next actions. (RPS/Dil).

  • The use of simulation-based assessment by the Royal College of General Practitioners in its Clinical Skills Assessment test. The aim of the CSA is to test a doctor’s ability to gather information and apply learned understanding of disease processes and person-centred care appropriately in a standardised context, make evidence-based decisions, and communicate effectively with patients and colleagues. Being able to integrate these skills effectively is a key element of this assessment. The validity of the CSA resides in its realistic simulation of real-life consultations. Patients are played by trained and calibrated role players, and cases are written and assessed by working GPs. The format of the assessment also allows for systematic sampling from the curriculum, using a selection blueprint. Each candidate is allocated a consulting room and has 13 ten minute consultations. (for details see:http://www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-clinical-skills-assessment-csa.aspx).

 (RPS)

Nursing

  • Simulations of the real world dilemmas that arise in practice identified by successful early nursing practitioners – tests the student’s ability to apply skills and knowledge to a unique situation, diagnosis, client relations and capacity to deliver and evaluate the results.

 (RPS/Dil)

  • Use of simulated patients to confirm ability to apply skills and knowledge effectively.

 (RPS)

  • Use of a video trigger in exam conditions to which the examinee must respond in terms of identifying what is happening, why, how the patient is behaving and what, in the light of this they would do to manage the situation, with a justification.

 (RPS)

  • Integrated assessment using video feedback. Entry-level nursing students in an accelerated nursing program complete a video project entitled “putting it all together”, in which they film themselves performing a sterile, dry dressing change on a volunteer patient. The videos are uploaded and shared on a secure server. This assignment allows students to demonstrate their ability to perform a sterile, dry dressing change integrating three distinct components of the course: health assessment, relational skills and psychomotor skills. In addition, students view a classmate’s performance and provide a constructive online peer review. After receiving the peer review, students summarize the learning by completing a short, online self-assessment, in which they reflect on their performance in the various components, how well they were able to combine the skills, and what their key learning was. (University of Toronto).

(RPS/Reflect)

Occupational Health and Safety

  • In a unit on critical thinking and decision-making students are given a scenario where they are supervising employees who are clearly doing something unsafe and illegal (e.g. entering a confined space tank). Students are asked to draw a flow chart of what is actually being done and a flow chart of what the law requires; then to compare the two. They are asked to draw a causal loop diagram of the non-technical issues going on (e.g. workers and/or employer are complacent). Students then must make a decision about what to do, referencing professional ethics and legal compliance, and draw up a communication plan for how they will communicate this decision to the people involved. The final report is a letter to the boss explaining what and why they are doing what they are doing.

 (Case/RPS)

Occupational Therapy

  • Students are presented with an online video of a scenario played by actors involving client family with a child who has cerebral palsy. Students have to identify what questions they will ask, what the optimum plan of action might be and give reasons to justify their approach based on what been learnt in the course.

(RPS/Case/PBL)

  • In a simulation-based assessment task students are given a specific role as part of an O.T. team and confronted with a case where there are number of possible ways to respond to the case as outlined. Students have to say what way of proceeding is likely to be most productive and why. This task gives focus to seeing how well the student would handle the most common real-world dilemmas that occur in early career practice.

 (RPS/Dil)

Policing

(RPS)

(RPS/ICT)

  • Simulated forensic science scenes. A range of clues are built into the scene and the student has to locate them and explain what they imply, referring to the key principles discussed in class.

 (RPS)

Psychology

  • A staged formative and summative assessment process is used in a fourth year class in this program. Each team of four meets with a ‘client’ - a trained psychologist who is role playing - and has to come up with an appropriate counselling response – with a series of dilemmas thrown in by the ‘client’ as they proceed. This is videoed – there is staff, peer and self review of the video. Each student is to identify what everyone said went well, what did not work well and what they would do next time, relating their evaluation to the input of the course. It is this capstone that confirms students are capable of undertaking a foundational counselling session. The room in which the simulation takes place replicates the real life context.

(Cap/RPS)

Science

  • Lab quality management – students check the lab for compliance against key quality assurance and safety checkpoints and then their work is evaluated by the external team of auditors they will actually have to satisfy when in the workforce.

(RPS/Field)

  • Use of triggers – science students are instructed on good practice in doing a presentation then they have to develop a presentation to the Board of a fictional company which is planning to close down its R&D department – this gets them to think through why they are doing science.

 (RPS/PBL)

Sustainable social, cultural, economic and environmental development

  • In a university-wide elective on Interdisciplinary Sustainable Development at the University of Manchester students are placed in teams and assigned a trained teamwork facilitator. Students are presented with a series of written project briefs and are given the role of sustainability consultants who must advice a series of clients on live, current problems, empowering them to make creative suggestions and think pragmatically how to devise an implementation plan that might work in practice… This requires them to balance economic, social and environmental consequences and take into account the ethics of the situation… Students are immersed in ‘wicked’ open-ended problems as discussed by Rittel and Webber in their 1973 book Dilemmas in a general theory of planning. Formative and summative team project reports are marked on the appropriateness of their response to the brief; the use of credible and relevant information; their development of a creative and well-justified proposal and their application of sustainability principles.

(For full details see Helen Dobson and Bland Tomkinson ‘Practical education for sustainable development through interdisciplinary problem-based learning’, University of Manchester Ch 3 in Richard Atfield and Patsy Kemp (Eds) (2013); Enhancing education for sustainable development in Business and Management, Hospitality, Leisure, Marketing, Tourism, HEA, York at : https://www.heacademy.ac.uk/sites/default/files/esd_dobson_final_0.pdf).

(RPS/PBL)

Theology

  • Business ethics and theology – different groups are given a position and they have to argue whether they support it or not. Each group has to discuss their conclusions with the other groups and identify the key points of difference and similarity, along with how convincing others found their argument to be. We reflect on what they have learnt and on the most effective learning approaches, on how to construct an argument, how to manage a project, the progression of argumentation, how to become self-reflective, self-evaluative learners. They then write a reflective report on the whole process.

 (Reflect/RPS)