Fair profiling with algorithms? Now there is a practical guide

Algorithms that discriminate, even when they are not designed to

Many public-sector organisations use so-called profiling algorithms: systems that decide, based on data, which citizens are subject to further review, checks or support. Examples include fraud detection and benefit allocation. These algorithms do not need to include variables such as ‘migration background’ or ‘gender’ to still exhibit bias on those dimensions. This is known as indirect discrimination.

Cor Veenman, Senior Scientist Specialist at TNO and researcher in Responsible AI, illustrates this with an example: ‘Suppose an algorithm flags people who frequently pay in cash as potential money launderers. At first glance, this seems neutral. But in some cultures, cash payments are simply more common in everyday transactions. The algorithm does not recognise that context, and still makes a distinction.’

Situations like this have occurred in the Netherlands in recent years. That is precisely why the NTA was developed: a Dutch Technical Agreement designed not only to assess problems afterwards, but above all to prevent algorithms from going wrong in the first place.

A safety net for algorithms outside the AI Act

If you assume the European AI Act already solves this issue, you are only partly right. The regulation focuses on high-risk AI systems based on machine learning. However, many algorithms used by governments fall outside its scope: they are rule-based, relatively simple, and are not self-learning. ‘The NTA is a safety net,’ says Veenman. ‘If you are not covered by the AI Act, you otherwise lack guidance. The NTA fills that gap.’

Voluntary, but practical

The NTA is not legislation, but a voluntary guideline issued by NEN. Because it can be developed more quickly than formal standards, organisations gain access to practical guidance sooner. The NTA for profiling algorithms is the result of more than a year of work involving 35 organisations: public bodies such as DUO, the Tax Administration and the Ministry of the Interior and Kingdom Relations, alongside research institutes, universities and civil-society organisations. Technical experts, lawyers and ethicists worked together throughout.

Technical expertise at the core

TNO played a specific role in this process. ‘We contributed technical expertise on AI systems and machine learning,’ says Veenman. ‘We looked at how to design algorithms, how to measure bias, and how to ensure effectiveness at the same time. An algorithm with zero bias that delivers no results also fails its purpose.’

Checklist for fair use: how the NTA works in practice

The NTA is a 65-page document that you can work through broadly as a checklist. It provides guidance across four areas: the purpose and context of the algorithm, the data and assumptions, testing and monitoring, and transparency and accountability.

Imagine you are a policymaker in a municipality and you want to use an algorithm to determine who is eligible for a specific provision. You work through the NTA step by step: have I clearly defined the purpose of this algorithm? What data am I using, and does it include variables that directly correlate with gender or background? Do I have the right expertise in place - legal, ethical, technical? And if I then carry out a bias assessment, what constitutes an acceptable outcome - and what does not?

Veenman highlights a key metric: ‘One of the minimum requirements is what we call demographic parity. If 10% of the population has a migration background, you would expect a similar proportion in the group selected by the algorithm. A significant deviation is a signal that something may be wrong, and you must be able to justify it or revise your approach.’

No guarantees, but a concrete starting point

The NTA provides guidance, but it does not guarantee non-discrimination. Responsibility remains with the organisation itself. ‘The NTA does not act as a referee,’ says Veenman. ‘You are responsible for doing the job properly. But it helps you understand what ‘proper’ means in this context.’

Its voluntary nature is deliberate. There is currently no specific legislation covering profiling algorithms outside the AI Act. Mandatory enforcement is therefore not an option and not the aim. ‘We want public organisations to use the NTA because it helps them, not because they have to,’ says Veenman. The fact that multiple government bodies actively contributed to its development is encouraging: ‘They were involved because they were waiting for this.’

Getting started with the NTA - with TNO support

Veenman hopes the NTA will be widely adopted, not only by national bodies but also by municipalities and other public-sector organisations. ‘If you are planning to use an algorithm, or already use one to advise, analyse or profile citizens: use the NTA to ensure you do so responsibly. From an ethical, legal and technical perspective.’

TNO can support organisations in applying the NTA. Whether by working through the checkpoints, conducting bias assessments or embedding the guideline into existing processes. The expertise TNO contributed during development is equally available for implementation.