[TMP-072] LWMs teaching to teach

Contact person: Agnes Grünerbl (agnes.gruenerbl@dfki.de) 

Internal Partners:

1. DFKI, EI, Agnes Grünerbl  

External Partners:

1. Health Department, Unviversity of Southampton, Eloise Monger  

High-quality education and training of nurses are of utmost importance to keep high standards in medical care. Nevertheless, as the covid pandemic has shown quite impressively, there are too few healthcare professionals available. Therefore, education and training of nurse students, or adapting the training of nurses is challenged to accelerate, to have manpower of nurses available when it is required. Still, accelerating training often comes with reduced quality, which can easily lead to bad qualifications and, in the worst case, to a lethal outcome. Thus, in nurse training a pressing question is, how to optimize and with it accelerate training without suffering in quality. One of the significant questions for teachers in training nurse students is to understand the state of a student’s education. Are some students in need of more repetitions? Which students can proceed to the next level, who is ready to get in contact with actual patients? In this regard, optimization of training means to individualize, not only individualize the training of students but also individualize the feedback and information a teacher gets about their way of teaching. We believe this to be a field where Artificial Intelligence (AI) and more specifically the application of foundational models (LLMs large language models, paired with other methods of machine learning) can provide real support. In the first part of this microproject, together with Nurse-Teachers of the University of Southampton, we want to define and design an LWM that fits the requirements of nurse training. For this, 2-3 nurse teachers from Southampton will visit DFKI in order to get a feeling for systems that are available, and also what applications are feasible. In turn, researchers of DFKI will visit the nurse training facilities in Southampton to get a better picture of how nurse training is conducted. At the end of this first phase of the microproject, an LWM (large whatever model) is defined (existing LLMs combined with additional features and data sources, as required). In the second phase, this LWM will be implemented and tested against videos of recorded training sessions. Specific focus will be set on:• How to understand the action of a particular person?• Actions taken by the trainee, are they correct or false? What would have been the correct action?• Which teaching efforts work and which do not as much? • Which useful suggestions and feedback can be provided to the trainees and teachers?

Results Summary

Building models of medical procedures require efforts that go beyond the scope and time frame of a micro-project. Therefore, this work is still ongoing and will proceed after the end of the Humane AI Net.

So in regards of project result at the time Humane AI Net ended is:

• identification of scenarios with a potential for generative AI to benefit health training – training of cannulation and venipuncture
• defining a procedure how to introduce Generative AI in training of cannulation and venipuncture.
• planning a study towards developing the required LWM models
• recording an extensive data-set in an actual medical training facility following actual training procedures.
• starting the long process of data processing and algorithm development (which is ongoing)

We collected a dataset consisting of: 90h of video (20 person recording 4 sessions of about 20+ min each, from 3 different cameras) acompanied with respective IMU Data + GoPro user view + audio recording and expert feedback of the process of cannulation and venipuncture.

Tangible Outcomes

1. [arxiv] Stefan Fritsch and Matthias Tschoepe and Vitor Fortes Rey and Lars Krupp and Agnes Gruenerbl and Eloise Monger and Sarah Travenna, GenAI Assisting Medical Training, arXiv, mobiCHAI workshop in MobileHCI2024 https://arxiv.org/abs/2410.16164 
2. presented at: mobiCHAI – 1st International Workshop on Mobile Cognition-Altering Technologies (CAT) using Human-Centered AI, at The ACM International Conference on Mobile Human-Computer Interaction Melbourne, Australia https://ai-enhanced-cognition.com/mobichai/