Numerous companies and organizations are currently exploring the implementation of AI applications. These applications are anticipated to not only simplify tasks and optimize the use of existing resources but to also become essential for maintaining future economic competitiveness. As a result, these entities are looking for advice on how to integrate AI applications while complying with data protection regulations.

The DSK guide primarily addresses AI applications with a particular focus on Large Language Models (LLMs) and is directed at those responsible for utilizing AI applications within their company or organization, thereby acting as data controllers. LLMs are frequently employed as chatbots but also serve as the foundation for other applications. Although the principles discussed herein are pertinent to various other AI models and applications, the guide does not specifically concentrate on the applications and training of AI models themselves.

Planning the use and selection of AI applications

First, the DSK guide offers guidance on the 'basic data protection principles' that must be observed when utilizing an AI application.

Fields of application and purposes

Prior to deploying AI applications, data controllers must explicitly define the intended fields and specific purposes of use. This is essential to ensure that the processing of personal data is necessary for the specified purposes, particularly for public authorities to ensure compliance with their legally mandated public tasks.

Lawfulness of fields of application

Certain applications of AI may be restricted by regulations, such as social scoring and real-time biometric monitoring under the EU Artificial Intelligence Act (AI Act), which are either entirely prohibited or permitted only under limited exceptions.

Non-personal data fields of application

Certain AI applications may not involve personal data. In such instances, data protection laws are not applicable. However, comprehensive assessments are required to ensure that no personal data is indirectly implicated. Consequently, the evaluation of whether personal data is present within a particular field of application must be conducted meticulously and continuously throughout the data's lifecycle.

Data protection-compliant training of AI applications

AI applications must be trained in accordance with data protection regulations. This includes verifying the legal basis for utilizing personal data in the training process and ensuring that training errors do not compromise compliant data processing.

Legal basis for data processing

Each stage of processing personal data through AI applications requires a legal basis, which differs depending on the context (e.g., public or private entities, specific domain). Public authorities may require specific legal grounds that address potential risks to the rights of data subjects.

No automated final decision-making

Pursuant to Article 22(1) of the EU General Data Protection Regulation (GDPR), decisions with legal effect must be made by humans, with AI providing only non-binding recommendations. Exceptions are permitted only in specific cases, such as when the data subject has given consent. If an AI application generates recommendations that have legal effects on a data subject, the process must ensure that the human decision-maker retains genuine discretion and that the decision is not predominantly based on the AI recommendation. The DSK further emphasizes that 'insufficient personnel resources, time pressure and a lack of transparency regarding the decision-making process of the AI-supported preliminary work must not lead to the results being accepted without scrutiny. The mere formal involvement of a human being in the decision-making process is not sufficient.'

Open or closed system?

AI applications can be classified as either open or closed systems. Open systems are typically operated by the provider as a cloud solution and accessible to an unspecified group of users via the internet. Consequently, input data exits the protected environment and, depending on the design of the AI application, may be used by one user to respond to queries from other users. This presents a risk of personal data being further processed for other purposes or being accessed and disclosed to unauthorized third parties. Additionally, data controllers must consider the potential transfer of data to third countries, a common occurrence in such setups. Open systems can also access other data sources, such as the internet, thereby creating personal data links or augmenting information about individuals.

Conversely, a closed system, as the name implies, processes data within a restricted and technically secured environment. This setup limits access to the AI application to a select group of users and ensures that control over the input and output data remains with those users. The system is designed so that the data entered or generated by the application is not used by the system provider for further training. Consequently, closed systems are preferable from a data protection perspective.

Transparency

Controllers must fulfill transparency obligations in accordance with Article 12 et seq. of the GDPR. While this may be straightforward when using a self-developed system, it becomes more challenging when the controller must rely on sufficient information from AI providers, particularly for cloud-based AI applications. The difficulty increases when using AI applications for automated decision-making. In such cases, transparency obligations include explaining the logic and scope of the automated decision-making process. Controllers must disclose information about the logic involved, including profiling as required by Article 22(1) of the GDPR, as well as the scope and potential impact on the data subject.

Transparency and choice regarding AI training

Users should be informed if their data is used for training AI and should have the option to exclude their data from such purposes. Legal bases are required if personal data is involved in training.

Transparency and choice regarding input history

Users must be informed about the storage of their input history and be able to decide whether to save it, particularly in shared environments.

Rectification, erasure, and other data subject rights

It is unsurprising that the data subject rights outlined in Chapter 3 of the GDPR are also applicable in the context of AI applications. However, the DSK underscores that controllers must ensure data subjects can effectively exercise their rights to rectification and erasure. This is particularly pertinent as AI applications may process inaccurate personal data for various reasons, necessitating correction or refinement through retraining. The DSK rightly critiques the inadequacy of many AI application providers (especially LLM chatbots), which merely advise users to verify results without guaranteeing accuracy. Consequently, the DSK asserts that AI applications should be engineered to uphold these rights, including addressing inaccuracies and facilitating permanent data deletion.

A recurring topic among data protection experts is the implementation of the right to erasure in connection with AI, especially considering the difficulty in tracing the precise processing of information by AI to produce results. While this guidance does not offer a definitive solution, the DSK clarifies why suppressing unwanted output through downstream filters typically does not constitute erasure within the meaning of Article 17 of the GDPR. This is because the data leading to a specific output after input may still be retained by the AI model in a personalized format. However, while not equivalent to rectification or erasure, filter technologies can assist in mitigating unwanted outputs.

Involvement of DPOs and employee representatives

Data protection officers (DPOs) and employee representatives should be involved in decisions regarding AI applications to ensure compliance and address concerns where needed.

Implementation of AI applications

In addition to discussing the basic data protection requirements, the DSK also comments on the data protection-compliant implementation of AI applications.

Define responsibilities and make binding arrangements

Under the GDPR, the entity responsible for determining the purposes and methods of processing personal data is designated as the controller. If an organization employs an AI application on its own servers, it assumes sole control. Conversely, utilizing an external provider's AI application constitutes data processing on behalf of the controller, necessitating an agreement pursuant to Article 28(3) of the GDPR. Additionally, joint controllership, as per Article 26 of the GDPR, may apply when two or more parties collaboratively determine the purposes and methods of processing. In such cases, a transparent agreement delineating each party's GDPR obligations and distinct legal bases for data processing is imperative.

Establish internal regulations

Establishing clear internal regulations is imperative to mitigate the risk of unauthorized employee use of AI applications, which may result in breaches of data protection regulations. These regulations should include comprehensive guidance and concrete examples outlining permissible and prohibited uses, meticulously documented in writing to provide users with clear direction. The DSK further advises -irrespective of personal data processing- the implementation of a service/works agreement between management and staff representation.

DPIAs

Prior to initiating the processing of personal data, a comprehensive risk assessment must be conducted. If the processing is likely to present a high risk to the rights and freedoms of individuals, Article 35 of the GDPR mandates the completion of a Data Protection Impact Assessment (DPIA). While this may initially seem straightforward, as elaborated in §1.11, if the controller is not also the provider of the AI system, reliance on information from the provider becomes essential. Thus, it is imperative to ensure - before entering into a contract with a service provider - that pertinent information, particularly regarding the system's functionality, is provided by the provider to facilitate a thorough risk assessment or DPIA.

Protect employees, set up company accounts

Employers should provide devices and accounts designated for professional use of AI applications, steering clear personal account utilization, which can lead to the creation of employee profiles. Unless managed on the company's servers, accounts should refrain from featuring individual names, and instead, functional email addresses should be employed.

Data protection through technology design and default settings

In accordance with Article 25 of the GDPR, controllers are obligated to adhere to the requirements of data protection by design and data protection by default, necessitating the implementation of data protection principles through technical and organizational measures, taking into account the specificities of AI systems. For instance, features enabling the use of input for AI training and retaining input history should be deactivated by default.

Data security

AI applications are required to adhere not only to Article 25 of the GDPR but also to overarching data protection legislation, encompassing technical and organizational measures outlined in Article 32 of the GDPR. These measures aim to guarantee confidentiality, integrity, availability, and resilience. Unauthorized access to AI applications can potentially compromise both personal and business information.

Raising awareness among employees

Employees should be educated on the appropriate and permitted use of AI applications.

Observe further developments

Data controllers are obligated to remain up to date with and adhere to both technical and legal advancements impacting the processing of personal data and risk mitigation. This encompasses meeting new obligations introduced by the AI Act and consistently assessing and adjusting internal standards as integral components of data protection management (Article 24 of the GDPR).

Use of AI applications

In conclusion, the DSK elaborates further on the utilization of AI applications and outlines additional considerations beyond those mentioned earlier.

Caution when inputting and outputting personal data

Data subjects should receive transparent information, including disclosure of the legal basis for both processing and transferring data to AI providers when the input data is personal data. Particularly, in AI applications aimed at generating cross-references from unstructured data, simply eliminating names and addresses does not suffice to de-identify data, as the context may still unveil personal information. Additionally, outputs generated by AI can also constitute personal data, necessitating a legal basis and potentially triggering notification to data subjects under Article 14 of the GDPR.

Take extra caution with special categories of personal data

Special categories of personal data (e.g., health, religious beliefs) are afforded significant protection under Article 9 of the GDPR and typically necessitate specific conditions for processing. For instance, AI employed in medical diagnostics must adhere to professional standards and be duly authorized as a medical device pursuant to Article 9(2)(h) of the GDPR, or alternatively, obtain explicit consent as outlined in Article 9(2)(a) of the GDPR.

Check results for accuracy

Results from AI applications must be critically examined for accuracy, especially when personal data is involved.

Check results and procedures for discrimination

Even if AI results are factually accurate, it is imperative to scrutinize them for potential discriminatory effects. Discriminatory outcomes, such as gender bias in hiring recommendations, can result in unauthorized data processing under the GDPR and breaches of Equal Treatment Acts, for example, the German General Equal Treatment Act (AGG).

Conclusion

To summarize, in its objectives, the DSK guide is in line with those of other European data protection authorities. The DSK takes all data protection principles into account but, as expected, is reluctant to make specific statements on implementation and execution under data protection law in relation to AI applications. However, as this guide is a living document according to its own disclaimer and will certainly be filled with more specific proposals for action in the future, it is worth keeping an eye on it.

Daniela Schott Attorney at Law
[email protected]
Kristin Bauer Attorney at Law
[email protected]
KINAST Rechtsanwaltsgesellschaft mbH
Attorneys at Law, Germany