We will develop a conceptual model of key components relating to supporting healthy behavior change. The model will provide a top-level representation of the clinical (from the psychological perspective) enablers and barriers that can be exploited for developing fine-grained models supporting the realization of behavior change paths within and across specific domains.

The resulting ontology will form the basis for generating user models (Theory of Mind), developing reasoning and decision-making strategies for managing conflicting values and motives, which can be used in collaborative and persuasive dialogues with the user. Such knowledge is also fundamental for embedding empathic behavior as well as non-verbal behaviors which can be embodied by a virtual character in the role of a coach. Learning methods can be applied to explore trajectories of behavior change. The produced ontology will represent a valuable resource for the healthcare domain thanks to the knowledge included into the provided resource.

Output

1 conference paper containing the description of the ontology and guidelines for its usage

1 ontology artifact

Presentations

Project Partners:

• Fondazione Bruno Kessler (FBK), Mauro Dragoni
• Centre national de la recherche scientifique (CNRS), Jean-Claude Martin
• Umeå University (UMU), Helena Lindgren

Primary Contact: Mauro Dragoni, FBK

Main results of micro project:

This micro-project led to three tangible outputs:
– 1 conference paper presented at the AI4Function workshop of IJCAI titled "An Ontology-Based Coaching Solution for Increasing Self-Awareness of Own Functional Status".
– 1 journal paper currently under revision at the Data Intelligence Journal (MIT Press) titled "Modeling a Functional Status Knowledge Graph
For Personal Health".
– 3 extensions of the HeLiS ontology publicly released and accessible at https://horus-ai.fbk.eu/helis/
Concerning the last point, the work done in this microproject worked as trigger for the creation of a new knowledge graph of functional status.

Contribution to the objectives of HumaneAI-net WPs

The resulting ontology will form the basis for generating user models (Theory of Mind), developing reasoning and decision-making strategies for managing conflicting values and motives, which can be used in collaborative and persuasive dialogues with the user. Such knowledge is also fundamental for embedding empathic behavior as well as corresponding non-verbal behaviors which can be displayed by a virtual character embodying the role of a coach or the dispatching of motivational human-computer interactions over different devices (e.g. mobile phone and smartwatch).
The micro-project relates to WP6 (T6.5 and AI coach for behavior change), WP3 (T3.4 User models and interaction history, and empathy in human-AI systems) and WP5 (explanation of medical advice, conflicting values and moral limits of nudging).

Tangible outputs

• Publication: An Ontology-Based Coaching Solution for Increasing Self-Awareness of Own Functional Status – Mauro Dragoni
  http://ceur-ws.org/Vol-2926/paper1.pdf
• Dataset: HeLiS extensions – Mauro Dragoni
  https://horus-ai.fbk.eu/helis/

Attachments

3.11 The Knowledgeable Coach FBK CNRS UMU pitch-video_Berlin.mkv

With the rise of social media platforms, we have witnessed the emergence of alarming phenomena in public debates, such as the polarization of opinions. The Friedkin-Johnsen model is a very popular model in opinion dynamics, validated on real groups, and well-investigated from the opinion polarization standpoint. Previous research has focused almost exclusively on static networks, where links between nodes do not evolve over time. However, it has been shown that they can break down if they are not strong enough to sustain disagreement.

In this micro-project, we want to fill this gap by designing a variant of the Friedkin-Johnsen model that embeds the dynamicity of social networks. Furthermore, we will design a novel definition of global polarization that combines network features and opinion distribution, to capture the existence of clustered opinions. We will analyse the polarization effect of the new dynamic model, and identify the impact of the network structure.

Output

Code of the simulator for the new dynamic model

Working paper, to be published in a workshop (possibly conference)

Project Partners:

• Consiglio Nazionale delle Ricerche (CNR), Elisabetta Biondi
• Central European University (CEU), Janos Kertesz
• Central European University (CEU), Gerardo Iniguez

Primary Contact: Elisabetta Biondi, CNR-IIT

Collective intelligence and a deliberative democratic system rest on the shoulders of a public free access to unbiased and diverse information. Social media has become a source of news and an important factor in opinion-formation. However, social media is not a neutral terrain but can be a platform for political manipulation.

This project aims to understand the cognitive, socio-affective, and neurobiological factors impacting our judgement of and response to information.This can conclude to behavioural recommendations that empower us to protect ourselves from manipulation and disinformation, and a targeted design of interventions to prevent their spread and the effective allocation of resources to protect those most vulnerable.

Output

Paper (aiming for Communication, Nature Human Behavior or similar)

Project Partners:

• ETH Zurich (ETHZ), Elisabeth Stockinger
• Fondazione Bruno Kessler (FBK), Riccardo Gallotti

Primary Contact: Elisabeth Stockinger, ETH Zurich

Current speech translation data sets contain pre-segmented speech audio, post-processed transcripts, and reference translations. Such data do not allow identifying error contributions of individual components in the whole speech translation pipeline and often lack detailedness to identify major error contributors. There is also lack of data for evaluating speech translation of web-based meetings (Zoom, Microsoft Teams, etc.) that have become vital to enable remote work and remote international cooperation.

To address the gaps, we propose:

1. to collect a data set of multi-speaker covering various domains and languages.

2. to create multi-layer annotations of the data that would allow evaluating individual components of pipeline-based (and also end-to-end) speech translation systems and measure each component's contribution towards the total error.

The data will feature audio data from multiple languages (including Latvian and Lithuanian for which no speech translation evaluation sets have ever been created).

Output

Data set for evaluation of speech translation systems consisting of audio data (in at least 3 languages)

Multi-layer annotations of the data (with at least the following annotation layers – speaker segmentation, sentence segmentation, orthographic transcriptions, normalised transcriptions, translations), and documentation of the data.

Project Partners:

• Charles University Prague, Ondrej Bojar

Primary Contact: Aivars Bērziņš, Tilde

This second micro project of WP6.10 will validate the air quality prototype developed in the first micro project with a second, real city, Valladolid in Spain. In principle, there will be no new developments except for feedback about the system from the city. This project is also in line with the objective of Humane AI: to shape the AI revolution in a direction that is beneficial to humans both individually and societally, and that adheres to European ethical values and social, cultural, legal, and political norms. The focus of the project will be on insights generated from data collected with mobile air quality measurement stations to be placed on top of vehicles, and to test how does insides help local governments in better managing the challenges around air quality.

Output

A report with the results of the evaluation of the prototype.

Adaptations to the system based on feedback from the local city.

Dissemination activities jointly by Telefonica and the local city.

Potential policy measures to improve the air quality in Valladolid.

Project Partners:

• Telefónica Investigación y desarrollo S.A. (TID), Richard Benjamins

Primary Contact: Richard Benjamins, Telefonica (TID)

This micro-project aims to explore the integration of virtual coaching and the use of Automated Dietary Monitoring wearable devices by conducting a preliminary study in which these technologies are integrated and experimented in a real-life context. Such a capability will enable the investigation about the role of real-time sensing data within a healthcare monitoring system supporting users by suggesting the most appropriate healthy behavior. The designed strategy will be validated within a living lab involving a group of 20-30 users that will wear the textile and will use the application for a period of three weeks.

Goals are to validate the capability of the smart textile in detecting chewing activities in a real-world environment, the effectiveness of the coaching system in detecting unhealthy dietary behaviors, the quality of the feedback provided, and the overall acceptability of the users with respect to a coaching system introducing a minimum invasive strategy.

Output

1 conference paper containing the description of the proposed approach together with the results and the insights gathered from the living lag we run.

1 dataset containing all the generated data that will be made available to the HumanE-AI network

Project Partners:

• German Research Centre for Artificial Intelligence (DFKI), Paul Lukowicz
• Fondazione Bruno Kessler (FBK), Mauro Dragoni

Primary Contact: Mauro Dragoni, FBK

The basic challenge regarding debiasing ML models is that in order to prevent models from generating bias on the basis of some sensitive characteristics it is necessary to have information about these characteristics. Usually this information is not available Fortunately, there is a new approach: Adversarial Reweighted Learning which debiases the models without having sensitive attribute information. However this approach redefines the fairness as Rawlsian max-min principle which is quite different from parity based fairness definitions that have been hitherto used. The goal of this project is to scrutinize the implications of using Rawlsian fairness principle in order debias the models by scrutinizing three things

1. Are there sensitive attributes for which Rawlsian fairness is unsuitable?

2. What would be the parity-based fairness scores when Rawlsian fairness definition is used for debiasing the models?

3. Is it possible to use Rawlsian fairness (and ARL) as post-processing method to existing models?

Output

Publication on identifying the effect of Rawlsian fairness on parity-based fairness definitions

Publication on how ARL can be used for models that are already being used as a post-processing tool

Project Partners:

• ING Groep NV, Dilhan Thilakarathne

Primary Contact: Dilhan Thilakarathne, ING

Blockchains such as Bitcoin and Ethereum currently are computational wasteful. On an annual basis both blockchains consume over a 70 terawatt-hours (TWh) of energy on computational resources to secure the network, which is similar to the annual energy consumption of Switzerland. These computational resources are used to reverse a cryptographic hash function (which is called a consensus algorithm) that is a solution to a puzzle, but serves no other purpose. Such amount of computational resources should be used more efficiently. Our aim is to use the large amount of computational resources more efficient by replacing the cryptographic hash function with a machine learning task. We focus on the Ethereum network as its computational power is not limited to solving hash functions only, as is mostly the case in Bitcoin. However, our intended solution can be generalized to any blockchain that is currently a computational wasteful.

Output

Publication: Review paper

Project Partners:

• ING Groep NV, Dilhan Thilakarathne

Primary Contact: Dilhan Thilakarathne, ING

Algebraic Machine Learning (AML) offers new opportunities in terms of transparency and control. However, that comes along with many challenges regarding software and hardware implementations. To understand the hardware needs of this new method it is essential to analyze the algorithm and its computational complexity. With this understanding, the final goal of this microproject is to investigate the feasibility of various hardware options particularly in-memory processing hardware acceleration for AML.

Output

Simulation model for a PIM architecture using AML

Report

Presentations

Project Partners:

• Algebraic AI S.L., Fernando Martin Maroto
• Technische Universität Kaiserslautern (TUK), Christian Weis
• German Research Centre for Artificial Intelligence (DFKI), Matthias Tschöpe

Primary Contact: FERNANDO MARTIN MAROTO, Algebraic AI

Main results of micro project:

We have carried out a theoretical study of the AML sparse crossing algorithm efficiency and identified in-memory processing and FPGA combined with in-memory processing as the two feasible options for Algebraic Machine Learning. Currently, we are working on a prototype implementation that involves FPGA and in-memory processing of bit arrays in commercial Upmem RAM memories.

Contribution to the objectives of HumaneAI-net WPs

This work is critical to speed up the calculation of Algebraic Machine Learning models and in so doing contribute to:
1- Bidirectional human-machine communication using formal expressions
2- Possibility to set goals and establish limits via formal constraints
3- Reduced dependency on statistics can help overcome bias
4- Transparency by design
5 -Possibility for decentralized, cooperative distributed machine learning

Tangible outputs

• Program/code: AML engine prototype using bitarrays – Fernando Martin Maroto
  www.algebraic.ai

This micro project will study the adaptation of automatic speech recognition (ASR) systems for impaired speech. Specifically, the micro-project will focus on improving ASR systems for speech from subjects with dysarthria and/or stuttering speech impairment types of various degrees. The work will be developed using either German “Lautarchive” data comprising only 130 hours of untranscribed doctor-patient German speech conversations and/or using English TORGO dataset. Applying human-in-the-loop methods we will spot individual errors and regions of low certainty in ASR in order to apply human-originated improvement and clarification in AI decision processes.

Output

Paper for ICASSP 2021 and/or Interspeech 2022

Presentations

Project Partners:

• Brno U, Mireia Diez
• Technische Universität Berlin (TUB), Tim Polzehl

Primary Contact: Mireia Diez Sanchez, Brno University of Technology

Main results of micro project:

Project has run for less than 50% of its allocated time.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Contribution to the objectives of HumaneAI-net WPs

WP1 Learning, Reasoning and Planning with Human in the Loop
T1.1 Linking symbolic and subsymbolic learning

WP3 Human AI Interaction and Collaboration
T3.1 Foundations of Human-AI interaction and Collaboration
T3.6 Language-based and multilingual interaction
T3.7 Conversational, Collaborative AI

WP6 Applied research with industrial and societal use cases
T6.3 Software platforms and frameworks
T6.5 Health related research agenda and industrial usecases

Tangible outputs

• Publication: –
• Other: Internal report – Mireia Diez Sanchez, mireia@fit.vutbr.cz

Understanding the mechanism of the neural correlates during human physical activities is important for providing safety in industrial factory environments considering brain activity during lifting a weight. Moreover, different responses to the same task can be observed due to physiological and neurological differences among individuals. In this project, the change pattern in EEG will be investigated during lifting of a weight and the features in EEG data making difference during lifting a weight will be analyzed. Classification between lifting and no lifting cases will be realized by using deep learning based machine learning methods. The outcomes of the project can be applied in industrial exoskeleton applications as well as physical rehabilitation of stroke patients.

Output

Dataset Repository (Share on AI4EU)

Conference Paper / Journal Article

Presentations

Project Partners:

• Türkiye Bilimsel ve Teknolojik Araştırma Kurumu (TUBITAK), Sencer Melih Deniz
• German Research Centre for Artificial Intelligence (DFKI), Paul Lukowicz

Primary Contact: Sencer Melih Deniz, TUBITAK BILGEM

Main results of micro project:

The project has run for almost 50% of its allocated time and has yet to be completed. Within this time duration, the following steps were completed:
1. Experimental paradigm was designed to achieve the project goals.
2. Study preparation including hardware and software development was completed.
3. Data recording session has been started and is in progress. Data from a total of 10 people has been obtained so far. More participants will be included in data acquisition to achieve the desired result.

The dataset and results will be evaluated once the data acquisition is completed.

Contribution to the objectives of HumaneAI-net WPs

This project is also part of WP2 with task numbers T2.2, T2.3.

This project aims to contribute to WP2 and WP6 by investigating the use case of EEG signal and AI models in the detection of various aspects of physical activities during weightlifting. To investigate pattern change in EEG during weightlifting will be aimed at providing more information in prediction of intended and actual human actions during sensori-motor tasks. Doing so, a common research question is aimed to be applied to the more industrial use cases such as control of exoskeletons. Moreover, outcomes of the project can be used for contribution in increasing mobility in stroke patients and disabled people as related with healthy living and mobility.

Tangible outputs

• Program/code: Data Acquisition Software Code – Juan Felipe Vargas Colorado

Attachments

PresentMovie_NeuralMech_WLlifting_TUBITAK_DFK_Berlin.m4v

Knowledge discovery offer numerous challenges and opportunities. In the last decade, a significant number of applications have emerged relying on evidence from the scientific literature. ΑΙ methods offer innovative ways of applying knowledge discovery methods in the scientific literature facilitating automated reasoning, discovery and decision making on data.

This micro-project will focus on the task of question answering (QA) for the biomedical domain. Our starting point is a neural QA engine developed by ILSP addressing experts’ natural language questions by jointly applying document retrieval and snippet extraction on a large collection of PUBMED articles, thus, facilitating medical experts in their work. DFKI will augment this system with a knowledge graph integrating the output of document analysis and segmentation modules. The knowledge graph will be incorporated in the QA system and used for exact answers and more efficient Human-AI interactions. We will primarily focus upon scientific articles on Covid-19 and SARS-CoV-2.

Output

Paper(s) in a conference or/and journal

Demonstrator

Presentations

Project Partners:

• ATHINA, Haris Papageorgiou
• German Research Centre for Artificial Intelligence (DFKI), Georg Rehm

Primary Contact: Haris Papageorgiou, ATHENA RC/Institute for Language & Speech Processing