[TMP-017] AI-Assistant to Mitigate Confirmation Bias in Cooperative Bayesian Optimization

Contact person: Samuel Kaski,, Aalto University (Samuel.kaski@aalto.fi) 

Internal Partners:

1. Aalto University, Samuel Kaski, samuel.kaski@aalto.fi
2. Delft University of Technology (TU Delft), Frans Oliehoek. F.A.Oliehoek@tudelft.nl

This micro-project contributes to developing methodologies that allow humans to be interactively involved “in the loop”. Here, the loop is a cooperative Bayesian optimization game where the goal is to optimize a 2D black-box function. At each iteration, the AI chooses the first coordinate, and then the user observes and opts for the second. Finally, the function is queried and the result is shown to both parties. The researcher can control agents’ characteristics, making it suitable for studying confirmation bias and imperfect knowledge. The project investigates how a planning AI agent can alleviate BO regret due to the human agent’s biases and imperfect information allocation. The aim is to build a planning AI agent to aid the user in the optimization task, where no single party has full decision-making power.

Results Summary

In this mini-project, we conducted an experiment with a synthetic user for various scenarios. We assumed the user decision process comprises two hierarchical steps:

updating the belief and taking action based on the belief. In the user model, these steps are regulated by model parameters, determining the conservatism in belief updates and exploration in opting for actions. Regarding that AI-assistant has a well-specified user model, we formulate the assistance decision problem as an MDP with unknown parameters ? and ?. We adjusted the Bayes-adaptive Monte Carlo planning methods to our problem to find the best policy for AI.

The reward in our planning AI is a weighted sum of two parts. Intuitively, one is responsible for ensuring that the AI favours actions for which the user can choose a promising second coordinate, and the other is responsible for reducing the risk that the user will act suboptimally, especially when the user model fails to predict the user well.

We compared our planning AI’s performance to a greedy AI (GreedyAI), which only tries to optimize the function based on its updated knowledge without considering the user. We also considered random query optimization (RandOpt) and a BO algorithm with the same acquisition function as GreedyAI uses but with full access to the data as logical lower and upper bounds for the optimization performance, respectively. The results demonstrate that the planning AI can assist the user in optimizing the function significantly better (measured as BO regret) than the GreedyAI and RandOpt, and relatively close to the logical upper bound under some conditions, even with its imperfect information.

Interestingly, a well-designed reward makes the cooperation effective even when the user follows a relatively high explorative policy. Investigating in-depth reveals that the planning AI lets the user explore the function adequately and reduces the chance of getting stuck at local optima.

In summary, the findings indicate that integrating a user model into a planning AI agent can mitigate potential biases of the user, enabling the team to avoid local optima and achieve better planning outcomes in sequential optimization games. By anticipating the user’s future behaviour, the agent can better guide the user towards optimal query.

Tangible Outcomes

1. Khoshvishkaie, A., Mikkola, P., Murena, P. A., & Kaski, S. (2023, September). Cooperative Bayesian optimization for imperfect agents. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 475-490). Cham: Springer Nature Switzerland. https://link.springer.com/chapter/10.1007/978-3-031-43412-9_28