Contact person: Sebastian Feger (sebastian.feger@um.ifi.lmu.de)

Internal Partners:

1. LMU, Sebastian Feger, sebastian.feger@um.ifi.lmu.de
2. GE, Jan Alpmann, alpmann@german-entrepreneurship.de

External Partners:

1. European Laboratory for Learning and Intelligent Systems (ELLIS)
2. ETH Zurich AI center

Connecting the right people with the right expertise is essential for innovation, but finding suitable collaborators can be challenging. In this micro project, we explored a novel approach that leverages publicly available performance data (e.g., Github profiles) to automatically match people and expertise for innovation. Building on such data, we developed a recommender system that suggests potential collaborators based on their demonstrated expertise.

The project builds on the previous two micro projects, which identified a set of functional questions to match people for innovation and implemented a prototype for creating user profiles based on those questions. In this third micro project, we developed a demonstrator that creates recommendations for collaboration based on the automated collection and analysis of GitHub data. In contrast to the previous micro projects, the approach focuses on demonstrated expertise, rather than relying on self-reported knowledge, which can be inaccurate or incomplete.

Results Summary

To develop the demonstrator, we used the GitHub API to retrieve information about users’ activities and built a recommender system that suggests potential collaborators based on their GitHub statistics. We also created a user interface for founders to find potential co-founders based on the expertise shown. This system offers a reliable and objective way of matching people with the right expertise, which can be especially valuable in high-tech environments where expertise is critical for success.

We evaluated the effectiveness and usability of the developed recommender system and user interface through a series of interviews and online studies. First, we interviewed domain experts to establish the requirements and important features of the model. Second, we interviewed potential founders to evaluate the general approach and gather feedback on the recommender system. Finally, we conducted an online study to evaluate the usability of the user interface. The study involved a group of participants using the interface to find potential collaborators and provide feedback on their experience. Overall, the evaluations provided valuable insights and feedback for improving the system’s effectiveness and usability. We plan to incorporate the feedback from the evaluations to further improve the system and enhance its ability to match people and expertise for innovation in the future. The user-centered design process benefited from close collaboration and interaction with experts working at external partners ELLIS and the ETH Zurich AI Center. The long-term goal is to help connect experts and entrepreneurs across physical boundaries, creating a vibrant and agile high-tech environment on a European scale.

Tangible Outcomes

1. Functional prototype with real-world data from github: http://143.42.16.26:3000/ 

Contact person: Agnes Grünerbl (agnes.gruenerbl@dfki.de) 

Internal Partners:

1. DFKI, EI, Agnes Grünerbl  

External Partners:

1. Health Department, Unviversity of Southampton, Eloise Monger  

High-quality education and training of nurses are of utmost importance to keep high standards in medical care. Nevertheless, as the covid pandemic has shown quite impressively, there are too few healthcare professionals available. Therefore, education and training of nurse students, or adapting the training of nurses is challenged to accelerate, to have manpower of nurses available when it is required. Still, accelerating training often comes with reduced quality, which can easily lead to bad qualifications and, in the worst case, to a lethal outcome. Thus, in nurse training a pressing question is, how to optimize and with it accelerate training without suffering in quality. One of the significant questions for teachers in training nurse students is to understand the state of a student’s education. Are some students in need of more repetitions? Which students can proceed to the next level, who is ready to get in contact with actual patients? In this regard, optimization of training means to individualize, not only individualize the training of students but also individualize the feedback and information a teacher gets about their way of teaching. We believe this to be a field where Artificial Intelligence (AI) and more specifically the application of foundational models (LLMs large language models, paired with other methods of machine learning) can provide real support. In the first part of this microproject, together with Nurse-Teachers of the University of Southampton, we want to define and design an LWM that fits the requirements of nurse training. For this, 2-3 nurse teachers from Southampton will visit DFKI in order to get a feeling for systems that are available, and also what applications are feasible. In turn, researchers of DFKI will visit the nurse training facilities in Southampton to get a better picture of how nurse training is conducted. At the end of this first phase of the microproject, an LWM (large whatever model) is defined (existing LLMs combined with additional features and data sources, as required). In the second phase, this LWM will be implemented and tested against videos of recorded training sessions. Specific focus will be set on:• How to understand the action of a particular person?• Actions taken by the trainee, are they correct or false? What would have been the correct action?• Which teaching efforts work and which do not as much? • Which useful suggestions and feedback can be provided to the trainees and teachers?

Results Summary

Building models of medical procedures require efforts that go beyond the scope and time frame of a micro-project. Therefore, this work is still ongoing and will proceed after the end of the Humane AI Net.

So in regards of project result at the time Humane AI Net ended is:

• identification of scenarios with a potential for generative AI to benefit health training – training of cannulation and venipuncture
• defining a procedure how to introduce Generative AI in training of cannulation and venipuncture.
• planning a study towards developing the required LWM models
• recording an extensive data-set in an actual medical training facility following actual training procedures.
• starting the long process of data processing and algorithm development (which is ongoing)

We collected a dataset consisting of: 90h of video (20 person recording 4 sessions of about 20+ min each, from 3 different cameras) acompanied with respective IMU Data + GoPro user view + audio recording and expert feedback of the process of cannulation and venipuncture.

Tangible Outcomes

1. [arxiv] Stefan Fritsch and Matthias Tschoepe and Vitor Fortes Rey and Lars Krupp and Agnes Gruenerbl and Eloise Monger and Sarah Travenna, GenAI Assisting Medical Training, arXiv, mobiCHAI workshop in MobileHCI2024 https://arxiv.org/abs/2410.16164 
2. presented at: mobiCHAI – 1st International Workshop on Mobile Cognition-Altering Technologies (CAT) using Human-Centered AI, at The ACM International Conference on Mobile Human-Computer Interaction Melbourne, Australia https://ai-enhanced-cognition.com/mobichai/

Contact person: John Shawe-Taylor, UCL (j.shawe-taylor@ucl.ac.uk) 

Internal Partners:

1. University College London í(UCL), John Shawe-Taylor
2. Institut “Jožef Stefan” (JSI), John Shawe-Taylor
3. INESC TEC, Alipio Jorge  

Through this work, we explore novel and advanced learner representation models aimed at exploiting learning trajectories to build a transparent, personalised and efficient automatic learning tutor through resource recommendations. We elaborate on the different types of publicly available data sources that can be used to build an accurate trajectory graph of how knowledge should be taught to learners to fulfil their learning goals effectively. Our aim is to capture and utilise the inferred learner state and the understanding the model has about sensible learning trajectories to generate personalised narratives that will allow the system to rationalise the educational recommendations provided to individual learners. Since an educational path consists heavily of building/following a narrative, a properly constructed narrative structure and representation is paramount to the problem of building successful and transparent educational recommenders.

Results Summary

Adding humanly-intuitive model assumptions to the TrueLearn Bayesian learner model such as 1) interest, 2) knowledge of learner 3) semantic relatedness between content topics has been achieved successfully leading to improved predictive performance. A dataset of personalised learning pathways of over 20000 learners has been composed. Analysis on Optimal Transport for generating interpretable narratives using Earth Mover’s Distance (EMD) of Wikipedia concepts also showed promise in scenarios where there is a limited number of topic annotations per document. A novel method for cross-lingual information retrieval using EMD has been invented pursuing this idea. Incorporating semantic networks (WordNet, WikiData) in building higher-level reasoning for recommendation also shows promise albeit with limited results at this point. Successful expansion of WordNet network using WikiData network is achieved. The resultant semantic network indicates that the quality of reasoning over Wiki Annotated video lectures can be improved in this way.

Tangible Outcomes

1. X5Learn: A Personalised Learning Companion at the Intersection of AI and HCI – Maria Perez-Ortiz. https://dl.acm.org/doi/10.1145/3397482.3450721  
2. “Why is a document relevant? Understanding the relevance scores in cross-lingual document retrieval.” Novak, Erik, Luka Bizjak, Dunja Mladenić, and Marko Grobelnik. Knowledge-Based Systems 244 (2022): 108545. https://dl.acm.org/doi/10.1016/j.knosys.2022.108545
3. Towards an Integrative Educational Recommender for Lifelong Learners (Student Abstract). Sahan Bulathwela,  María Pérez-Ortiz, Emine Yilmaz, and John Shawe-Taylor. (2020, April). In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 10, pp. 13759-13760). https://ojs.aaai.org/index.php/AAAI/article/view/7151/7005
4. “TrueLearn: A Python Library for Personalised Informational Recommendations with (Implicit) Feedback.” Yuxiang Qiu,  Karim Djemili, Denis Elezi, Aaneel Shalman, María Pérez-Ortiz, and Sahan Bulathwela.  arXiv preprint arXiv:2309.11527 (2023). Published through ORSUM workshop, RecSys’23 https://arxiv.org/pdf/2309.11527
5. “Peek: A large dataset of learner engagement with educational videos.” Bulathwela, Sahan, Maria Perez-Ortiz, Erik Novak, Emine Yilmaz, and John Shawe-Taylor.  arXiv preprint arXiv:2109.03154 (2021). Submitted to ORSUM workshop, RecSys’21 https://arxiv.org/abs/2109.03154
6. Dataset: PEEK Dataset – Sahan Bulathwela https://github.com/sahanbull/PEEK-Dataset
7. Program/code: TrueLearn Model – Sahan Bulathwela https://github.com/sahanbull/TrueLearn
8. Program/code: Semantic Networks for Narratives – Daniel Loureiro https://github.com/danlou/mp_narrative

Contact person: Mohamed Chetouani (mohamed.chetouani@sorbonne-universite.fr) 

Internal Partners:

1. ISIR, Sorbonne University, Mohamed Chetouani, Silvia Tulli
2. Vrije Universiteit Amsterdam, Kim Baraka  

Human-Interactive Robot Learning (HIRL) is an area of robotics that focuses on developing robots that can learn from and interact with humans. This educational module aims to cover the basic principles and techniques of Human-Interactive Robot Learning. This interdisciplinary module will encourage graduate students (Master/PhD level) to connect different bodies of knowledge within the broad field of Artificial Intelligence, with insights from Robotics, Machine Learning, Human Modelling, and Design and Ethics. The module is meant for Master’s and PhD students in STEM, such as Computer Science, Artificial Intelligence, and Cognitive Science. This work will extend the tutorial presented in the context of the International Conference on Algorithms, Computing, and Artificial Intelligence (ACAI 2021) and will be shared with the Artificial Intelligence Doctoral Academy (AIDA). Moreover, the proposed lectures and assignments will be used as teaching material at Sorbonne University, and Vrije Universiteit Amsterdam. We plan to design a collection of approximately 12 1.5-hour lectures, 5 assignments, and a list of recommended readings, organized along relevant topics surrounding HIRL. Each lecture will include an algorithmic part and a practical example of how to integrate such an algorithm into an interactive system. The assignments will encompass the replication of existing algorithms with the possibility for the student to develop their own alternative solutions. Proposed module contents (each lecture approx. 1.5 hour): (1) Interactive Machine Learning vs Machine Learning – 1 lecture, (2) Interactive Machine Learning vs Interactive Robot Learning (Embodied vs non-embodied agents) – 1 lecture, (3) Fundamentals of Reinforcement Learning – 2 lectures, (4) Learning strategies: observation, demonstration, instruction, or feedback- Imitation Learning, Learning from Demonstration – 2 lectures- Learning from Human Feedback: evaluative, descriptive, imperative, contrastive examples – 3 lectures, (5) Evaluation metrics and benchmarks – 1 lecture, (6) Application scenarios: hands-on session – 1 lecture, and (7) Design and ethical considerations – 1 lecture.

Contact person: John Shawe-Taylor (j.shawe-taylor@ucl.ac.uk)

Internal Partners:

1. Knowledge 4 All Foundation, Davor Orlic, davor.orlic@gmail.com
2. University College London, John Shawe-Taylor, j.shawe-taylor@ucl.ac.uk
3. Institut Jožef Stefan, Davor Orlic and Marko Grobelnik, davor.orlic@gmail.com

K4A proposed a microproject to extend its existing prototype of the online learning platform X5LEARN (https://x5learn.org/) developed by K4A and UCL and JSI and its new IRCAI center under the auspices of UNESCO. It is a standalone, learner-facing web application designed to give access through an innovative interface to a portfolio of openly licensed educational resources (OER) in video and textual format. Designed for lifelong learners looking for specific content wanting to expand on their knowledge, our aim is to extend it to AI-related topics. The updated application will be released via IRCAI, a newly designated AI center and integrated with AI4EU with heavy HumaneAI branding. The main reason to push the product with IRCAI is that UNESCO is positioning itself as the main UN agency to promote humanist Artificial Intelligence, a major international policy on the Ethics of AI, and champion OER, which is in line with HumaneAI.

Results Summary

Under this microproject, a series of extensions to the X5Learn platform was added. A new user friendly user interface was developed and deployed. X5Learn, being an intelligent learning platform, a series of human-centric AI technologies that enable educational recommendation, intelligent previewing of information and scalable question generation that can help different stakeholders such as teachers and learners were developed backed by scientific research. The results have been published in peer reviewed conferences such as AAAI, AIED and CHIIR and also published in the Journal of Sustainability. The new earning platform is now available to the public including a python library that implements the recommendation algorithms developed.

Tangible Outcomes

1. Maria Pérez Ortiz, Sahan Bulathwela, Claire Dormann, Meghana Verma, Stefan Kreitmayer, Richard Noss, John Shawe-Taylor, Yvonne Rogers, and Emine Yilmaz. 2022. Watch Less and Uncover More: Could Navigation Tools Help Users Search and Explore Videos? In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval (CHIIR ’22). Association for Computing Machinery, New York, NY, USA, 90–101. https://doi.org/10.1145/3498366.3505814 
2. Maria Perez-Ortiz, Claire Dormann, Yvonne Rogers, Sahan Bulathwela, Stefan Kreitmayer, Emine Yilmaz, Richard Noss, and John Shawe-Taylor. 2021. X5Learn: A Personalised Learning Companion at the Intersection of AI and HCI. In 26th International Conference on Intelligent User Interfaces – Companion (IUI ’21 Companion). Association for Computing Machinery, New York, NY, USA, 70–74. https://doi.org/10.1145/3397482.3450721 
3. Sahan Bulathwela, María Pérez-Ortiz, Emine Yilmaz, and John Shawe-Taylor. 2022. Power to the Learner: Towards Human-Intuitive and Integrative Recommendations with Open Educational Resources. Sustainability 14, 18: 11682. https://doi.org/10.3390/su141811682 
4. [arxiv] Bulathwela, S., Pérez-Ortiz, M., Holloway, C., & Shawe-Taylor, J. (2021). Could ai democratise education? socio-technical imaginaries of an edtech revolution. arXiv preprint arXiv:2112.02034.https://arxiv.org/abs/2112.02034 
5.  X5Learn Platform: https://x5learn.org/ 
6.  TrueLearn Codebase: https://github.com/sahanbull/TrueLearn 
7.  TrueLearn Python library: https://truelearn.readthedocs.io 
8.  X5Learn Demo Video: https://youtu.be/aXGL05kbzyg 
9.  Longer lecture about the topic: https://youtu.be/E11YUWad7Lw 
10.  Workshop presentation (AAAI’21): https://www.youtube.com/watch?v=gYtmL2XdxHg 
11.  Workshop Presentation (AAAI’21): https://youtu.be/4v-fizLvHwA