AI Oversight Lab

We conduct joint research, guide development trajectories, and provide a safe space for learning and reflection. Interdisciplinary collaboration is central to our approach, because we believe ethical, organisational and technical aspects must be addressed together to create valuable and responsible AI.

You can work with us on:

• Evaluating algorithms and (generative) AI models for robustness, reliability and bias
• Responsible design and development of algorithms and AI systems
• Structured decision‑making on goals, values, risks and resources using our BVRR method
• Setting up governance structures and ethical frameworks
• Applying standards and regulatory frameworks in practice
• Assessing and strengthening AI maturity within your organisation
• Tailored training courses and workshops on responsible AI

We conduct interdisciplinary research on cross‑government challenges such as scaling AI, translate scientific insights into practice (and vice versa), and support the translation of scientific and practical insights into policy.

We help with:

• Developing guidelines and frameworks
• Trend analyses

By sharing experiences and best practices, we make scientific knowledge more accessible and foster a network built on learning, reflection and collaboration.

The challenge

The Immigration and Naturalisation Service (IND) developed a risk model to identify violations by recognised sponsors more quickly. The organisation wanted to be certain that the model was reliable and free from unwanted bias. It was also important to understand which factors influenced the outcomes and how inspectors could use the model responsibly.

The solution

AIOL held discussions with inspectors, managers and developers to gain insight into work processes and model usage. In joint workshops, we mapped potential forms of bias and carried out technical analyses of how the model handled sensitive variables such as nationality. This combination of practical insight and technical assessment made it possible to clearly identify risks.

The result

The findings were included in a report shared with the House of Representatives. IND acted on the recommendations and, together with AIOL, conducted follow‑up research into the alignment between the risk model and operational practice.

The challenge

The municipality of Nissewaard used an algorithm to select potential social‑benefit fraud cases. The key question was whether the model could be used responsibly: did it operate reliably, fairly and transparently? The municipality wanted clarity before continuing to use the model.

The solution

AIOL assessed the model against existing guidelines for responsible AI use. We interviewed municipal staff and the developer to understand how the model was built and applied. The technical evaluation focused on three aspects: whether the model was verifiable, whether it behaved as intended, and whether it fits well within the way of working of the organisation. This made it possible to clearly identify the main risks.

The result

The evaluation showed that the algorithm could not be used responsibly at that time. The municipality therefore decided to stop using it. The study underlined the importance of critical evaluation and careful consideration before deploying AI applications.

The challenge

The State Supervision of Mines (SodM) is exploring the use of generative AI to support inspectors in desk research. It is essential that any systems developed are responsible, reliable and valuable. The challenge was to evaluate the AI system holistically in order to assess whether it adds value and can be deployed responsibly.

The solution

Together with SodM, AIOL developed a holistic evaluation methodology based on the seven principles of responsible AI. Each principle was assessed at different levels to create an overall picture of the performance. An application developed by SodM was evaluated both technically and with users.

The result

SodM gained insight into how different aspects of responsible AI can be incorporated and assessed during the development of generative AI applications. This enables SodM to design a process for well‑founded decision-making about further development and deployment of AI applications.

The challenge

SodM is running several pilots to explore the potential of AI. To enable responsible AI use on a structural basis and to accelerate innovation, organisational AI maturity is essential. Moving from pilot to successful adoption across the organisation is not straightforward. The organisation needs to be prepared at multiple levels, such as AI literacy and AI governance. In addition, AI systems must be and remain compliant with legislation and other (internal) requirements.

The solution

Together with SodM, AIOL developed an AI maturity and innovation capacity scan, providing insight into the various aspects required to enable responsible AI use and accelerate innovation. We also mapped how legal, regulatory and organisational requirements can be monitored across different types of AI applications.

The result

Based on the maturity and innovation capacity scan, SodM can determine where to invest in order to facilitate responsible AI use across the organisation. This allows the organisation to seize AI opportunities more quickly and responsibly, while enabling innovation to progress.