16.9 REPAIR Dialogues with Isabella Banks

PhD Candidate Isabella Banks (Institute of Information Law, University of Amsterdam) will deliver an online presentation titled ‘Learning-by-doing’: Lessons on human oversight of AI systems from inside the Dutch judiciary on September 16, from 9:00 to 10:00 (EET).

Attend the session via Zoom: https://helsinki.zoom.us/j/69690804255?pwd=MnBDTODhWaNO2H4XvcrMaV2CtRPzff.1

‘Learning-by-doing’: Lessons on human oversight of AI systems from inside the Dutch judiciary

The EU AI Act makes human oversight central to the governance of high-risk AI systems, including those used by or on behalf of judicial authorities. It is also an important pillar of many of the judicial guidelines for the ethical use of technology that have emerged in recent years, which consistently identify human oversight as a safeguard against the risks to core judicial values that AI systems pose and suggest that it can be operationalized in various ways. The Dutch judiciary’s 2025 AI strategy – which establishes the institution’s intention to experiment with low-risk systems as part of ‘learning-by-doing’ approach to the ‘responsible and innovative’ use of AI – gives human oversight a similarly central role. Yet both the strategy and European legal and ethical frameworks stop short of identifying who should be responsible for overseeing AI systems used in the judiciary, when and how they should intervene, and what their oversight should aim to achieve.

This qualitative study aims to shed light on these domain- and system-specific practicalities by exploring the question, ‘How does the Dutch judiciary operationalize human oversight in relation to its AI pilot projects?’ through semi-structured interviews with individuals involved in developing, deploying, and overseeing three judicial AI systems. These include a system that pseudonymizes judicial decisions, an internal chatbot, and an online dispute resolution platform. The study describes the layered control measures through which these systems are overseen and, through comparative analysis, identifies early lessons that emerge from the Dutch judiciary’s experimentation with AI and human oversight.

Isabella Banks is a PhD candidate at the University of Amsterdam’s Institute for Information Law (IViR) and a member of the Public Values in the Algorithmic Society (AlgoSoc) research consortium. Her socio-legal research aims to understand how human oversight of AI systems used in the justice sector is conceptualized, organized, and governed. Isabella has a background in social science and criminology and previously conducted research at The Hague Institute for Innovation of Law (HiiL), the Public International Law & Policy Group (PILPG), and the Center for Justice Innovation.