On September 27, 2024, the Dutch data protection authority (AP) requested feedback on manipulative and exploitative artificial intelligence (AI) systems.

The AP highlighted that manipulative and deceptive AI systems and exploitative AI systems are two forms of prohibited AI under Articles 5(1)(a) and 5(1)(b) of the EU Artificial Intelligence Act (the AI Act) respectively. The AP reminded that provisions on prohibited AI practices under Article 5 of the AI Act enter into force on February 2, 2025.

Articles 5(1)(a) and 5(1)(b) of the AI Act concern the placing on the market, the putting into service, or the use of an AI system that:

• deploys subliminal techniques beyond a person's consciousness or purposefully manipulative or deceptive techniques, with the objective, or the effect of materially distorting the behavior of a person or a group of persons by appreciably impairing their ability to make an informed decision, thereby causing them to take a decision that they would not have otherwise taken in a manner that causes or is reasonably likely to cause that person, another person, or group of persons significant harm; or
• exploits any of the vulnerabilities of a natural person or a specific group of persons due to their age, disability or a specific social or economic situation, with the objective, or the effect, of materially distorting the behavior of that person or a person belonging to that group in a manner that causes or is reasonably likely to cause that person or another person significant harm.

The AP noted that to fall under either of the above provisions, the practice must be based on AI. While 'subliminal' and 'manipulative' techniques under Article 5(1)(a) of the AI Act may involve the use of components such as audio, images, or video stimuli that persons cannot perceive or control.

Regarding Article 5(1)(b) of the AI Act specifically, the AP added that the exploitation must relate to age, disability, or specific economic or social circumstances, with 'disability' defined by the EU Accessibility Requirements for Products and Services Directive.

In addition, the AP detailed that a condition for either prohibition is that the disruption ultimately leads or can reasonably lead to significant damage being suffered by a person or persons. This damage is noted to include significant negative consequences for physical or psychological health or financial interests.

Further, the AP clarified that the aim of the AI system must have the purpose or effect under Article 5(1)(a) of the AI Act, or the purpose or consequence under Article 5(1)(b) of the AI Act, of substantially disrupting the behavior of persons. Where the disruption is the result of factors outside the AI system over which the provider or deployer has no influence, it may be assumed there was no intention to disrupt behavior. The AP provides that for Article 5(1)(a) of the AI Act a significant disruption is considered to occur if the purpose or effect of an AI system noticeably impedes the ability of individuals to make an informed decision, causing them to make a decision that they would not otherwise have made. The AP also stipulated that although there is no information on what a disruption of behavior would be considered under Article 5(1)(b) of the AI Act, the bar for the application of such prohibition is considered to be lower because of the exploitation of vulnerabilities.

The prohibitions under Article 5(1)(a) and 5(1)(b) of the AI Act may also apply if damage is caused or is reasonably likely to occur to a group of persons.

Regarding the scope of the prohibitions, the AP emphasized, referencing the explanatory memorandum of the AI Act, that they should not affect AI systems in the context of medical treatment or common legitimate commercial practices conducted in accordance with relevant laws.

Public comments may be submitted until November 17, 2024.

You can read the press release here and the policy document here, both only available in Dutch.