KEYNOTE SPEAKERS

Lambert Schuwirth is a Strategic Professor in Medical Education, College of Medicine and Public Health at Flinders University.

He graduated from Maastricht Medical School as an MD and he has been involved in medical education and medical education research since 1990. His main interest is in assessment of medical competence and performance, both in undergraduate and postgraduate training settings. He worked at Maastricht University for almost 20 years as assistant, associate and full professor in the Department of Educational Development and Research, before coming to Flinders in August 2011.

In 1991, he joined the Department of Educational Development and Research at Maastricht Medical School, taking up various roles in student assessment: Chairman of the Inter-university and the Local Progress Test Review Committee, the OSCE Review Committee and the Case-based Testing Committee. Since the early 2000s, he has been Chair of the overall Taskforce on Assessment. He has been an advisor on assessment to medical colleges in the Netherlands, Australia and the UK. In 2010, he chaired an international consensus group on educational research, the results of which were published in Medical Teacher.

Assessment reform in a time of disruptive technological changes

Funnily enough, the technological development that is causing most disruption to health professions education (HPE) and assessment is one that converts numbers into words, or limited data into more data – generative rather than categorising artificial intelligence. It is tempting, then, to just look at this technology and make reactive changes or, even worse, to try to forbid the use of it in assessment in the hope that it will go away. But for an industry to manage disruption it is better to carefully examine its value proposition than merely looking at its processes. So, the “Why are we here, what value do we add to our students and society?”, rather than merely “What do we do?”.

Fundamentally, the emergence of generative AI in itself is not the major disruption, it is just the final trigger. Because information technology has already been changing society, and student and teacher affordances for over two decades. It has led to a democratisation of knowledge, the formation of distributed trust, the accessibility to cognitive surplus, the increased accessibility of IT affordances, and free access to an exploding array of generative artificial intelligence applications. A decrease or sometimes even reversal of expertise asymmetry between healthcare professional and patient/client and between teacher and student is an important consequence.

It has a particular impact on assessment in HPE, because in modern HPE, the value propositions of education and that of assessment are seriously misaligned. While the former aims to produce lifelong learners, collaborators, managers of uncertainty and complexity, and professionals who can turn ‘not knowing’ into ‘knowing’ as self-regulated learners, traditional assessment programs often only reinforce finite modules, competition, certainty and linearity, and professionals who just ‘know’. And where professional practice often embraces the use of technology from the perspective of distributed cognition, traditional assessment abhors it and focuses mainly on the student’s ‘biological’ competence and sees using technology as cheating or cognitive offloading.

So, technological developments have shone a brighter light on this misalignment and have made students and staff more aware of the downsides of the traditional assessment approach, such as McNamara’s fallacy, Goodhart’s law, reductionism, deficiency instead of diversity thinking and the poverty of reductionist feedback.

Assessment for learning programs – and not just formative feedback in an otherwise traditional assessment program – appear to be a way out, but they will require a drastic – and difficult – rethinking and redesign of assessment programs.

What the design principles for such programs would be, will be highlighted in this key note presentation.

Marcy Rosenbaum is Professor of Family Medicine and Faculty Development Consultant for the Office of Consultation and Research in Medical Education at the University of Iowa Carver College of Medicine. She has been actively involved in teaching, curriculum development and conducting research on clinician-patient communication and health professions education for the more than 30 years.

She oversees communication skills training for students, residents and practicing health care providers at the University of Iowa Hospitals and Clinics. She also has spent her career conducting research and directing programs focused on enhancing health professional faculty teaching skills in classroom and clinical settings. She has published extensively and facilitated train the trainers courses throughout the world. She is the current past president of EACH: International Association for Communication in Healthcare and past founder and Chair of tEACH, the teaching committee of EACH.

Exploring the complexity of communication skills and feedback (formative assessment) for healthcare learners

Health professional educators note that providing feedback (formative assessment) to learners on their performance that is well received and impacts learners’ subsequent performance is a consistent challenge. This talk will explore the unique challenges that accompany feedback conversations related to learners’ clinical communication skills as part of formal communication learning sessions and in the immediate context of learners’ interactions with patients during supervised patient care (including as part of assessments such as WBAs/Mini CEX). Questions to be considered will include if and how communication skills feedback is different from feedback on other clinical skills and why and when communication skills feedback is challenging to accomplish effectively, particularly in the context of workplace-based education. Assumptions underlying approaches to giving effective feedback on learner communication skills and also research challenges in investigating practices and outcomes of communication skills feedback will be critically examined. This talk will be of interest to anyone who is involved in communication skills education and/or assessment for healthcare professional learners whether as a classroom or clinical teacher, a curriculum and evaluation developer, a healthcare practitioner and/or as a healthcare education or practice researcher.

Professor Iain Martin is Vice-Chancellor and President of Deakin University in Australia.

He came to Deakin from the position of Vice-Chancellor of Anglia Ruskin University in the United Kingdom. Prior to that, he was Deputy Vice-Chancellor Academic at the University of New South Wales. Professor Martin spent a number of years at the University of Auckland in New Zealand with positions including Professor of Surgery, Dean of the Faculty of Medical and Health Science, and Deputy Vice-Chancellor with responsibility for external and strategic partnerships.

Professor Martin grew up in the United Kingdom and attended the University of Leeds where he completed his medical degree, Doctorate and Master of Education.

Since being appointed as Deakin’s Vice-Chancellor in 2019, he has served as Chair of the Barwon Regional Partnership and Chair of the Australian Technology Network (ATN) of Universities. Since January 2022, he has served as Chair of the Victorian Vice-Chancellors’ Committee and is a member of the CSIRO’s Australian Centre for Disease Preparedness Advisory Group.

In March 2023, Professor Martin was reappointed as Deakin’s Vice-Chancellor for a further five-year term and continues to lead the implementation of the University’s strategic plan, Deakin 2030: Ideas to Impact.

Generative AI and assessment in health professional education - Moral panic, ethics, wisdom and technology, what wins out?

The very rapid evolution of AI, especially the use and adoption of transformer based large language models is touching every aspect of education and assessment.

ChatGPT very publicly highlighted their spread with the platform reaching 100 million users in less than 2 months. The last months of 2022 and 2023 saw what can only be described as panic in many educational settings as to their impact on education, most notably assessment.

From the perspective of a university educational environment this talk will explore how we consider AI and in particular LLM’s on how we frame, design and deliver meaningful, valid and accurate assessment in the health professional education context, when these tools are ubiquitous.

The moral panic that has been evident given that these LLM based tools can ‘pass’ professional exams in medicine and law, should perhaps best be reframed as why is there not moral panic that we are relying on assessments that the currently available LLM’s can ‘pass’.

This talk will deliberately aim to be provocative and challenging in exploring the interface of assessment design, ethics, wisdom and technology as we embrace what may be possible, what should be done and how do we know what our students know.

Suzanne Schut is an Assistant Professor in Educational Science at Delft University of Technology. After graduating, she started working as a teacher and assessment consultant in higher education. In 2014, she joined the medical education community and completed her PhD research on programmatic assessment at the School of Health Professions Education (SHE), Maastricht University. Her main research interest is in adaptive expertise and workplace-based assessment, more specifically in teacher-student relationships within the assessment environment. Her activities in the Department of Educational Development and Research were focussed on student assessment. She chaired the assessment review committee of the MD program and is currently a member of the assessment expert panel of the World Health Organization (WHO). Over the years, she was privileged to work with many students, teachers, and other more knowledgeable partners, gaining a lot of experience with the design and implementation of assessment programs and facilitated faculty development courses on assessment and feedback in different settings and countries. Recently, she made the transition to the field of teacher education, aiming to challenge and enhance her understanding of the impact of assessment on students and teachers by investigating assessment approaches and interpersonal assessment relationships in a different field and context.