[TMP-117] The Effect of Perceived Task Difficulty on Human Overreliance on AI

Contact person: Tony Zhang (zhang@fortiss.org)

Internal Partners:

1. Fortiss, Tony Zhang, zhang@fortiss.org
2. TUBITAK, Sencer Melih Deniz, sencer.deniz@tubitak.gov.tr

For AI to be useful in high-stakes applications, humans need to be able to recognize when they can trust AI and when not. However, in many cases, humans tend to over rely on AI, i.e. people adopt AI recommendations even when they are not adequate. Explanations of AI output are meant to help users calibrate their trust in AI to appropriate levels, but often even increase overreliance. In this microproject, we aim to investigate whether and how overreliance on AI and the effect of explanations depend on the perceived task difficulty. We ran an AI-assisted decision-making experiment where participants are asked to solve a series of decision tasks (particular type of task to be determined) with support by an AI model, both with and without explanations. Along with participants’ reliance behavior, we will measure perceived task difficulty for each task and participant through EEG data.

Results Summary

We found that people are able to rely appropriately on AI recommendations in our decision-making task when the decision is easy for people, resulting in complementary team performance. However, when the decision is difficult for people, they heavily over rely on AI recommendations, leading to non-complimentary team performance.

Common feature-based explanations had no statistically significant effect on the appropriateness of people’s reliance, neither for easy nor for difficult decisions. However, we did find hints that, conceptually, feature-based explanations could further improve the appropriateness of reliance for easy decisions, but not for hard decisions.

Tangible Outcomes

1. [under review]  a submission to the journal Behaviour & Information Technology (BIT), Special Issue on Human-Centered Artificial Intelligence with only final minor revisions pending
2. Dataset of EEG recordings corresponding to easy and difficult decisions:  https://www.ai4europe.eu/research/ai-catalog/decision-difficulty-eeg-dataset