Contesting Black-Box Decisions

Ensuring that decisions made by systems adhere to human socio-ethical principles.

The right to contest a decision that has consequences on individuals or the society is a well-established democratic right. In the European Union, the General Data Protection Regulation explicitly requires the means of contesting decisions made by algorithmic systems. Contesting a decision is not a matter of simply providing explanation, but rather of assessing whether the decision and the explanation are permissible against an externally provided policy. Albeit its importance, little fundamental work has been done on developing the means for effectively contesting decisions. In this micro-project, we will develop the foundations needed to integrate the contestability of decisions based on socio-ethical policy (e.g. Guidelines for Trustworthy Artificial Intelligence (AI)) into the decision-making system. This microproject will lay the basis for a line of research in contestability of algorithmic decision making by considering the overall ethical socio-legal aspects discussed in WP5 of the HumanE-AI-Net project. During the course of this microproject, we will achieve 3 objectives: 1) extend our work on formal language for socio-ethical values, concretised as norms and requirements; 2) conceptualise our feedback architecture which will monitor the predictions and decisions made by an AI system, check the predictions against a policy; and 3) a logic to evaluate black-box prediction based on formal socio-technical requirements by extending our previous work on monitoring and assessing decisions made by autonomous agents. The end result will be an agent architecture which contains 4 core components: i) a predictor component, e.g. a neural network, able to produce recommendations for a course of action; ii) a decision-making component, which decides if and which action the agent should take; iii) a utility component, influencing the decision-making component by ascribing a utility value to a potential action to be taken; and iv) a ‘governor’ component; able to reason and suggest the acceptance or rejection of recommendations made by a predictor component. During the microproject, we plan to first focus on compliance checking but ensure our architecture is flexible and modular enough to facilitate extensions such as the governor component offering feedback for ‘retraining’ to the predictor component.

For the successful implementation of the project, we have devised the following working plan: we will first review relevant literature on government agents, and then proceed with the formalisation of ethical socio-legal policies. In parallel, we will start working towards a case example implementation to deliver a practical demo at the end. During the microproject, we have planned two research visits: one in Uni. of Berghem in June and one in Open University of Cyprus in October. Online weekly meetings take place between those visits.

Output

1. Use-case demo consisting of an agent making recommendations on a to-be-defined topic. The agent will be built using the architecture conceptualised in this micro project.
2. We plan 2 papers, one focusing on the theoretical implementation and one on the use-case implementation, targeting the following venues: AAAI, AAMAS, IJCAI, KR, ECAI, AIES, HHAI.

Project Partners

Primary Contact

Andreas Theodorou, Umeå University (UmU)