Making sense of data is a main challenge in creating human understandable descriptions of complex situations. When data refer to process executions, techniques exist that discover explicit descriptions in terms of formal models. Many research works envisage the discovery task as a one-class supervised learning job. Work on deviance mining highlighted nonetheless the need to characterise behaviours that exhibit certain characteristics and forbid others (e.g., the slower, less frequent), leading to the quest for a binary supervised learning task.

In this microproject we focus on the discovery of declarative process models, expressed through Linear Time Temporal Logic, as a binary supervised learning task, where the input log reports both positive and negative behaviours. We therefore investigate how valuable information can be extracted and formalised into a “optimal” model, according to user-preferences (e.g., model generality or simplicity). By iteratively including further examples, the user can also refine the discovered models

Output

Paper to be submitted to relevant journal

Machine learning tool

Artificial data set

Presentations

Project Partners:

  • Fondazione Bruno Kessler (FBK), Chiara Ghidini
  • Università di Bologna (UNIBO), Federico Chesani

Primary Contact: Chiara Ghidini, FBK

Main results of micro project:

The microproject has produced so far two main results:
– A two-step approach for the discovery of temporal-logic patterns as a binary supervised learning problem, that is starting from a set of "positive traces" (execution traces whose behaviour we want to observe in the discovered patterns), and a set of "negative" traces (execution traces whose behaviour we do not want to observe in the discovered patterns). In detail, in the first step, sets of patterns (possible models) that accept all positive traces and discard as much as possible of the negative ones are discovered. In the second step, the model(s) optimizing one criterion, as for instance the generality or the simplicity, are selected among the possible discovered models.
– Two synthetic labelled ("positive" and "negative") event log datasets used for the synthetic evaluation of the proposed approach.

Contribution to the objectives of HumaneAI-net WPs

The results of the microproject mainly contribute to WP1 (Human-in-the-Loop Machine Learning, Reasoning and Planning). Indeed, on the one hand, the micro-project aims at leveraging machine learning techniques (sub-symbolic learning) to provide LTL patterns (symbolic representation) of a set of "positive" traces, while excluding the "negative" ones (T1.1). On the other hand, the micro-project is a first step towards including the human in the loop of the discovery of LTL patterns representing all and only the cases the human wants to represent (T1.3). The user could indeed iteratively refine the discovered patterns so as to be sure to include all the cases she is interested to include, while excluding all those cases that she wants to exclude.

Tangible outputs

Results Description

The microproject has produced three main results:
– A two-step approach for the discovery of temporal-logic patterns as a binary supervised learning problem, that is starting from a set of "positive traces" (execution traces whose behaviour we want to observe in the discovered patterns), and a set of "negative" traces (execution traces whose behaviour we do not want to observe in the discovered patterns). In detail, in the first step, sets of patterns (possible models) that accept all positive traces and discard as much as possible of the negative ones are discovered. In the second step, the model(s) optimizing one criterion, as for instance the generality or the simplicity, are selected among the possible discovered models.
– Two synthetic labelled ("positive" and "negative") event log datasets used for the synthetic evaluation of the proposed approach.
– A paper describing the approach, as well as the approach evaluation.

The microproject has produced three main results:
– A two-step approach for the discovery of temporal-logic patterns as a binary supervised learning problem, that is starting from a set of "positive traces" (execution traces whose behaviour we want to observe in the discovered patterns), and a set of "negative" traces (execution traces whose behaviour we do not want to observe in the discovered patterns). In detail, in the first step, sets of patterns (possible models) that accept all positive traces and discard as much as possible of the negative ones are discovered. In the second step, the model(s) optimizing one criterion, as for instance the generality or the simplicity, are selected among the possible discovered models.
– Two synthetic labelled ("positive" and "negative") event log datasets used for the synthetic evaluation of the proposed approach.
– A paper describing the approach, as well as the approach evaluation.

Publications

Chesani, F., Francescomarino, C. D., Ghidini, C., Grundler, G., Loreti, D., Maggi, F. M., Mello, P., Montali, M., and Tessaris, S. (2022). Shape your process: Discovering declarative business processes from positive and negative traces taking into account user preferences. In Almeida, J. P. A., Karastoyanova, D., Guizzardi, G., Montali, M., Maggi, F. M., and Fonseca, C. M., editors, Enterprise Design, Operations, and Computing – 26th International Conference, EDOC 2022, Bozen-Bolzano, Italy, October 3-7, 2022, Proceedings, volume 13585 of Lecture Notes in Computer Science, pages 217–234. Springer.

Chesani, F., Francescomarino, C. D., Ghidini, C., Grundler, G., Loreti, D., Maggi, F. M., Mello, P., Montali, M., and Tessaris, S. (2022). Shape your process: Discovering declarative business processes from positive and negative traces taking into account user preferences. In Almeida, J. P. A., Karastoyanova, D., Guizzardi, G., Montali, M., Maggi, F. M., and Fonseca, C. M., editors, Enterprise Design, Operations, and Computing – 26th International Conference, EDOC 2022, Bozen-Bolzano, Italy, October 3-7, 2022, Proceedings, volume 13585 of Lecture Notes in Computer Science, pages 217–234. Springer.

Chesani, F., Francescomarino, C. D., Ghidini, C., Loreti, D., Maggi, F. M., Mello, P., Montali, M., and Tessaris, S. (2022). Process discovery on deviant traces and other stranger things. IEEE Transactions on Knowledge and Data Engineering, pages 1–17. DOI: https://doi.org/10.1109/TKDE.2022.3232207

Links to Tangible results

Loan Approval1: dataset. https://drive.google.com/drive/folders/15BwG4PJq8iIMh9Sr9dpMXAYBY-qp7QDE?usp=sharing
Loan Approval2: dataset. https://drive.google.com/drive/folders/1fcJ8itzdMbNOjEAeV6nUEeI5B6__aB_c?usp=sharing
Discovery Framework: program/code https://zenodo.org/record/5158528
Experiments: https://github.com/stessaris/negdis-experiments/tree/v1.0

This project continues the collaboration between FBK and TUW about defeasible knowledge in description logics in the Contextualized Knowledge Repository (CKR) framework.

In applications, knowledge can hold by default and be overridden in more specific contexts. For example, in a tourism event recommendation system, events can appear as suggested to a class of tourists in a general context: in the more specific context of a particular tourist, preferences can be refined to more precise interests, which may override those at higher contexts.

Goal of this project is to enhance the answer set programming (ASP) based realization of CKR to deal with complex context hierarchies: we use an ASP extension recently proposed by TUW, ASP with algebraic measures, which allows for reasoning on orderings induced by the organization of defeasible knowledge. This collaboration will provide a prototype for reasoning over CKR hierarchies, but also an application for ASP with algebraic measures.

Output

Prototype implementation: realization of reasoning service for query answering over CKR with contextual hierarchies. The prototype will be made available in AI4EU platform.

Report on formalization: technical report and paper submission containing the defining the formal aspects of model selection for contextual hierarchies via ASP with Algebraic Measures and some initial evaluations in the prototype.

Presentations

Project Partners:

  • Fondazione Bruno Kessler (FBK), Loris Bozzato
  • TU Wien, Thomas EIter

Primary Contact: Loris Bozzato, FBK

Main results of micro project:

The goal of this project is to reason on complex contextualized knowledge bases using an answer set programming extension with algebraic measures and show the capabilities of this formalism.

The main formal contributions of this project are:
– an extension of the CKR contextual framework to reason about defeasible information over multi-relational contextual hierarchies.
– an ASP based modelling of multi-relational CKRs, where combination of model preferences is realized via algebraic measure expressions.
– an asprin based implementation of query answering in a fragment of multi-relational CKRs, extending the existing CKR datalog translation.
– a study of further capabilities of algebraic measures, showing the possibilities for reasoning on model aggregation.

Contribution to the objectives of HumaneAI-net WPs

The results of the MP are relevant for AI as they show a combination of non-monotonic contextualized DLs in the CKR framework and Logic Programming with numerical measures in weighted LARS.

With respect to the HumaneAI vision, the resulting framework provides a tool for representing, e.g., complex social structures and the contextualization of information relative to such social organizations. With respect to the WP1 objectives, the work combines different AI areas, and follows the direction of joining symbolic and numeric knowledge representation and reasoning methods with notions of uncertainty.

Tangible outputs

Through this work, we explore novel and advanced learner representation models aimed at exploiting learning trajectories to build an transparent, personalised and efficient automatic learning tutor through resource recommendations. We elaborate on the different types of publicly available data sources that can be used to build an accurate trajectory graph of how knowledge should be taught to learners to fulfil their learning goals effectively. Our aim is to capture and utilise the inferred learner state and the understanding the model has about sensible learning trajectories to generate personalised narratives that will allow the system to rationalise the educational recommendations provided to individual learners. Since an educational path consists heavily of building/following a narrative, a properly-constructed narrative structure and representation is paramount

to the problem of building successful and transparent educational recommenders.

Output

Paper on development of narrative representations for learning

Enhancements of the X5Learn portal for accessing Open Educational Resources (OER)

Visualisation software for landscapes of learning

User evaluations of software

Presentations

Project Partners:

  • University College London í(UCL), John Shawe-Taylor
  • Institut “Jožef Stefan” (JSI), John Shawe-Taylor
  • INESC TEC, Alipio Jorge

Primary Contact: John Shawe-Taylor, UCL

Main results of micro project:

Adding humanly-intuitive model assumptions to the TrueLearn Bayesian learner model such as 1) interest, 2) knowledge of learner 3) semantic relatedness between content topics has been achieved successfully leading to improved predictive performance. A dataset of personalised learning pathways of over 20000 learners has been composed and under review for publication. Analysis on Optimal Transport for generating interpretable narratives using Earth Mover's Distance (EMD) of Wikipedia concepts also showed promise in scenarios where there is a limited number of topic annotations per document. A novel method for cross-lingual information retrieval using EMD has been invented pursuing this idea. Incorporating semantic networks (WordNet, WikiData) in building higher-level reasoning for recommendation also shows promise albeit with limited results at this point. Successful expansion of WordNet network using WikiData network is achieved. The resultant semantic network indicates that the quality of reasoning over Wiki Annotated video lectures can be improved in this way.

Contribution to the objectives of HumaneAI-net WPs

The main contributions are towards WP1 and WP3. In terms of WP1, improvements made to TrueLearn contribute towards building a much richer representation of the learner in the educational recommender. This builds towards linking symbolic and subsymbolic AI where the representation is understood by AI while it can be easily translated to a humanly-intuitive narrative. The proposed online learning scheme connects with continual learning where the model constantly updates itself on a lifelong basis. EMD and semantic network enrichment also further contribute to human-in-the-loop machine learning by empowering the human user in the recommendation process with potential narratives of learning trajectories. The results also connect to ideas in WP3 as the recommender’s foundations are built on user modelling and exploiting user interaction history. Having already built a rich representation, the results present opportunities to further contribute to identifying useful visualisations and human-AI collaboration frameworks to utilising the learner model in-the-wild.

Tangible outputs

  • Publication: X5Learn: A Personalised Learning Companion at the Intersection of AI and HCI – Maria Perez-Ortiz
    https://dl.acm.org/doi/10.1145/3397482.3450721
  • Publication: Understanding the Relevance Scores in Cross-Lingual Document Retrieval – Erik Novak
    Submitted to International Journal for Information Processing & Management
  • Publication: Towards Semantically Aware Educational Recommenders – Sahan Bulathwela
    To be Submitted.
  • Publication: TrueLearn IK: Modelling Interests and Knowledge of Online Learners – Sahan Bulathwela
    To be Submitted to EAAI'22
  • Publication: PEEK: A Large Dataset of Learner Engagement with Educational Videos – Sahan Bulathwela
    Submitted to ORSUM workshop, RecSys'21
  • Dataset: PEEK Dataset – Sahan Bulathwela
    https://github.com/sahanbull/PEEK-Dataset
  • Program/code: TrueLearn Model – Sahan Bulathwela
    https://github.com/sahanbull/TrueLearn
  • Program/code: Semantic Networks for Narratives – Daniel Loureiro
    https://github.com/danlou/mp_narrative

IRL, developed by Luc Steels and collaborators, is a parsing technique that captures the semantics of a natural language expression as a network of logical constraints. Determining the meaning of a sentence then amounts to finding a consistent assignments of variables that satisfies these constraints.

Typically, such meaning can only be determined (i.e. such constraints can only be resolved) by using the context ("narrative") in which the sentence is to be interpreted. The central hypothesis of this project is that modern large-scale knowledge graphs are a promising source of such contextual information to help resolve the correct interpretation of a given sentence.

We will develop an interface between an existing IRL implementation and an existing knowledge-graph reasoning engine to test this hypothesis. Evaluation will be done on a corpus of sentences from social-historical scientific narratives against corresponding knowledge graphs with social-historical data.

Output

Software: an interface between nat.lang. parsing software (IRL) and reasoning software (knowledge graphs)

Presentations

Project Partners:

  • Stichting VU, Frank.van.Harmelen@vu.nl
  • Universitat Pompeu Fabra (UPF), luc.steels@upf.edu

Primary Contact: Frank van Harmelen, Stichting Vrije Universiteit Amsterdam

Main results of micro project:

This micro-project aims to build a bridge between a language processing system (incremental recruitment language (IRL)) and semantic memory (knowledge graphs), for building and parsing narratives.
In IRL, a sentence is represented as a network of logical constraints. Resolving the interpretation of a sentence comes down to finding a consistent assignment of entities from the real world that satisfy these constraints. In this microproject, we have used knowledge graphs and other open data repositories as an external source of world knowledge that can be used to bind and disambiguate entities in context.

We have implemented a new library called Web-Services that interacts, through the use of APIs, with several open data knowledge repositories, and integrates their semantic facts into language models such as IRL. Using the Web-Services library, users can write IRL programs that send requests to different open data APIs, or convert SPARQL queries into RESTful APIs using GRLC.

Contribution to the objectives of HumaneAI-net WPs

Natural language processing and understanding in machines often relies on statistical pattern recognition.
What is missing here is the ability of a machine to describe in a human understandable way how it came to a certain interpretation.
This would allow humans to take part in a machine’s reasoning process, thereby facilitating human-computer interaction and collaboration.
By using IRL, the interpretation of an utterance is transparently expanded and ambiguous entities are resolved until a single interpretation is found. At the same time, large datasets with semantic knowledge about the world exist in open repositories on the web. These repositories could be used in a similar way as we humans use our semantic memory, to disambiguate entities that cannot be resolved using the context of a dialogue alone.

Tangible outputs

This project builds on earlier work by FBK in Trento on KENN <https://arxiv.org/pdf/2009.06087.pdf> and by VUA in Amsterdam <https://arxiv.org/abs/2006.03472> and aims to combine the insights of both. The project has 3 aims, depending on difficulty we may achieve one, two or all three.

1. The current version of KENN uses the Godel t-conorm. We will develop versions of KENN based on other t-conorms (like the product t-conorm and Łukasiewicz), whose properties have been investigated in the earlier work by VUA. This should improve the performance of KENN.

2. We will try to extend the expressivity of the logical constraints in KENN from sets of clauses to implications, again using the earlier theoretical work by VUA. This should increase the reasoning capabilities of KENN.

3. It be should possible to check the exact contribution of each clause to the final predictions of KENN. This will increase explainability of KENN.

Output

paper describing improvements to KENN, published in workshop or conference

software: new version of KENN

Presentations

Project Partners:

  • Stichting VU, Frank.van.Harmelen@vu.nl
  • Fondazione Bruno Kessler (FBK), serafini@fbk.eu

Primary Contact: Frank van Harmelen, Vrije Universiteit Amsterdam

Main results of micro project:

Project has run for less than 50% of its allocated time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Contribution to the objectives of HumaneAI-net WPs

Project has run for less than 50% of its allocated time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Tangible outputs

  • Other: –

Results Description

KENN is a neuro-symbolic architecture developed in Trento. It allows to inject a knowledge-base when training a neural network. Theoretical work from Amsterdam has been used to improve KENN. As a result of using background knowledge from a knowledge we can train the neural network with many fewer training examples. Since KENN is based on fuzzy logic, a major bottleneck was the choice of the appropriate configuration of the logic (choice of norms and co-norms), since earlier work from Amsterdam had showed that some of the classical fuzzy logic configurations would perform very poorly in a machine learning setting (with large areas of their value space having a 0 gradient, or a 0 gradient for one of their input values).
As a result of the collaborations (visit from Amsterdam staff to Trento and vice versa), we have developed so called Fuzzy Refinement Functions). Such "refinement functions" are functions that change the truth value computed by a fuzzy logic operator in order to improve the gradient behaviour, while still maintaining the desired logical combinatorics. We have implemented such refinement functions in an algorithm called Iterative Local Refinement (ILR). Our experiments have shown that ILR finds refinements on complex SAT formulas in significantly fewer iterations and frequently finds solutions where gradient descent can not. Finally, ILR produces competitive results in the MNIST addition task.

Publications

Refining neural network predictions using background knowledge,
Alessandro Daniele, Emile van Krieken, Luciano Serafini & Frank van Harmelen
Machine Learning (2023)

https://link.springer.com/article/10.1007/s10994-023-06310-3

Links to Tangible results

Publication at https://link.springer.com/article/10.1007/s10994-023-06310-3
Code and data at https://github.com/DanieleAlessandro/IterativeLocalRefinement

Online AutoML in environments where the working conditions change over time.

The main goal consists of studying online optimization methods for hyper-parameter tuning.

In dynamic environments, the “optimal” hyper-parameters might change over time.

Online AutoML will consist of an exploration phase followed by an exploitation phase.

The exploration phase is looking to find the set of hyper-parameters for the current working condition. The exploitation phase will continuously monitor the learning process to detect degradation in the performance of the system which triggers a new exploitation phase.

We will consider complex problems described by pipelines where each step in the pipeline has its own hyper-parameters. We will consider problems with many hyper-parameters where some of them might be irrelevant. Among the relevant parameters, the complexity of the model architecture (with particular reference to deep networks) is of particular relevance and will be the objective of our study.

Output

1 Conference Paper

1 Journal Paper

1 Prototype software

Presentations

Project Partners:

  • INESC TEC, Joao Gama
  • Consiglio Nazionale delle Ricerche (CNR), Giuseppe Manco

Primary Contact: Joao Gama, INESC TEC, University of Porto

Main results of micro project:

1) Paper accepted at ECMLPKDD 2021

Hyper-Parameter Optimization for Latent Spaces in Dynamic Recommender Systems
Bruno Veloso, Luciano Caroprese, Matthias Konig, Sonia Teixeira,
Giuseppe Manco, Holger H. Hoos, and Joao Gama

2) Prototype of an Online AUTOML tool available at github

3) A prototype of a data generator used in 1)

Contribution to the objectives of HumaneAI-net WPs

The main goal of this micro-project is to develop tools that help people use sophisticated machine learning algorithms by helping in the parameterization of these algorithms.

The micro-project involved three groups from 3 different countries:
INESC TEC, Portugal
ICAR-CNR, Italy
University Leiden, Netherlands

Based on this MP, the same groups will propose two new micro-projects.
News soon!

Tangible outputs

  • Publication: Hyper-Parameter Optimization for Latent Spaces in Dynamic Recommender Systems – Bruno Veloso, Luciano Caroprese, Matthias Konig, Sonia Teixeira,
    Giuseppe Manco, Holger H. Hoos, and Joao Gama
  • Program/code: Self Hyper-parameter tunning – Bruno Veloso, Luciano Caroprese, Matthias Konig, Sonia Teixeira,
    Giuseppe Manco, Holger H. Hoos, and Joao Gama
    https://github.com/BrunoMVeloso/ECMLPKDD2021
  • Dataset: Generator for preference data – Bruno Veloso, Luciano Caroprese, Matthias Konig, Sonia Teixeira,
    Giuseppe Manco, Holger H. Hoos, and Joao Gama
    https://github.com/BrunoMVeloso/ECMLPKDD2021

Involving human knowledge into the learning model can be done in diverse ways, e.g., by learning from expert data in imitation learning or reward engineering in deep reinforcement learning. In many applications, however, the expert data usually covers part of the search space or “normal” behaviors/scenarios. Learning a policy in the autonomous driving application under the limited dataset can make the policy vulnerable to novel or out of distribution (OOD) inputs and, thus, produce overconfident and dangerous actions. In this microproject, we aim to learn a policy based on the expert training data, while allowing the policy to go beyond data by interacting with an environment dynamics model and accounting uncertainty in the state estimation with virtual sensors. To avoid a dramatic shift of distribution, we propose to use the uncertainty of environment dynamics to penalize the policy for states that are different from human behavior.

Output

Paper https://2021.ieee-iv.org/ and/or

Paper http://www.auai.org/uai2020/

Yes

Project Partners:

  • German Research Centre for Artificial Intelligence (DFKI), Christian Müller
  • Volkswagen AG, Andrii Kleshchonok

Primary Contact: Christian Müller, DFKI

Main results of micro project:

The project has run for less than 50% of its allocated time. Until today, the partners have spent time refining the project goals. We intend to begin the actual project phase in November. We also had several meetings on the formulation of the project problem and possible approaches to solve it.

Contribution to the objectives of HumaneAI-net WPs

Our key idea in this project is to learn a representation with deep models in a way to incorporate rules (e.g., physics equations governing dynamics of the autonomous vehicle) or distributions that can be simply defined by humans in advance. The learned representations from the source domain (the domain whose samples are based on the defined equations/distributions) are then transferred to the target domain with different distributions/rules and the model adapts itself by including target-specific features that can best explain variations of target samples w.r.t. underlying source rules/distributions. In this way, human knowledge is considered implicitly in the feature space.

Tangible outputs

  • Other: TBD – TBD
    TBD