The AI Act takes a risk-based approach and regulates in particular the various AI systems’ risk classification and the requirements for high-risk AI systems and GPAI systems (general purpose AI). According to such approach, the following applies: the higher the risk of the application, the stricter the requirements. There is also a transparency obligation: artificially generated or edited content must be clearly labeled as such. The regulation affects users and operators of AI systems as well as providers, importers, retailers and manufacturers of AI systems.
Our Services for EU AI Act
- AI Act
- Compliance: Support in complying with the AI Act’s legal requirements, including the implementation of risk management and quality management systems
- Risk assessment: analyzing and assessing the risks that may arise from the use of high-risk AI systems
- Documentation and reporting: Assistance with the preparation of the necessary documentation and reports required under the AI Act
- Drafting of contracts: drafting and reviewing contracts in connection with the use and development of AI systems
- Data protection: Advice on data protection issues, in particular with regard to the processing of large amounts of data by AI systems
- IP law: protection of intellectual property generated by AI systems and advice on licensing issues
- Technology and IT law: support with legal issues relating to software development, IT outsourcing and cloud computing
What are AI systems and how are risks classified?
The legally compliant assignment of AI systems to one of the risk categories is relevant in the context of the AI Act and the AI Liability Directive. According to these regulations, an AI system is a machine-based system that is designed to operate with varying degrees of autonomy and can demonstrate adaptability after use – for explicit or implicit goals from the input it receives.
The AI Regulation distinguishes between several risk classes:
AI systems entailing an unacceptable risk are prohibited as they pose a threat to people. These include biometric real-time remote identification systems and AI systems that enable social scoring or manipulation techniques.
Furthermore, there are also high-risk AI systems. These pose a high risk to the health and safety or fundamental rights of people, but are permitted. Therefore, their development and use are subject to comprehensive documentation, monitoring and quality requirements. AI systems that pose a limited risk to the end user are systems intended for interaction with humans. They are subject to a limited number of transparency obligations. AI systems that are considered to pose a minimal risk are permitted without further ado.
What requirements apply in the case of high risk?
High-risk AI systems are divided into two categories: On the one hand, this includes AI systems that are used in products covered by EU product safety regulations. On the other hand, it also includes AI systems that are used in sensitive areas such as health, transport, justice or the police and which must be registered in an EU database. In the case of high-risk AI systems, there is a particular obligation to establish a risk management system, record-keeping and transparency, the creation of technical documentation, the obligation for human supervision and the obligation for these AI systems to achieve an appropriate level of accuracy, robustness and cybersecurity.
AI Governance
A central aspect of this category is AI governance. This describes the entirety of measures ensuring the ethical and legal use of artificial intelligence. Transparency, fairness and compliance with data protection regulations are crucial in this context.
For companies, this means that the functioning of their AI systems must be comprehensible in order to create trust among users and authorities. In addition, possible distortions in the algorithms must be actively prevented and security gaps avoided. This can only be achieved through regular employee training and clear guidelines. The right governance therefore plays a key role in ensuring that the benefits of AI can be exploited while risks remain under control.”
GPAI systems (General Purpose AI) are AI systems with a general purpose. Providers of GPAI systems must fulfill special requirements, such as the creation and updating of information and documentation as well as compliance with copyright law. GPAI systems are accessible to the general risk classification and are subject to the corresponding requirements.
AI and data protection - are they even compatible?
The solution is complex and feasible.
Irrespective of the AI Act, the use of artificial intelligence already requires compliance with all legal data protection regulations if personal data is processed with the AI.
The principles of the GDPR such as lawfulness, purpose limitation, transparency, data minimization and accuracy must be observed.
- Lawfulness means that data processing must be consistently based on legal grounds permitting the processing of personal data for specific purposes
- In particular, the informed nature of consent is doubtful due to the complexity of AI
- Information obligations pursuant to Art. 13, 14 GDPR
- Documentation – description of the AI system in the record of processing activities
- Inclusion of AI in the privacy policy
- Distinction between sole responsibility, processing or joint responsibility and corresponding contractual provisions
- Data protection by design and by default (Art. 25 GDPR)
- Technical and organizational measures taking into account the state of the art (Art. 32 GDPR)
- Raising employee awareness and conducting training courses
- Conducting a data protection impact assessment (DPIA) in accordance with Art. 35 GDPR: Blacklist of the German Data protection Conference, “DSK” - No. 11: The implementation of a DPIA is mandatory, among other things, when using AI in order to process personal data, to control the interaction with the data subjects or to evaluate personal aspects of the data subject
How will the AI regulation affect the world of work?
According to the AI Regulations, AI systems
- intended to be used for the recruitment or selection of natural persons (e.g., analysis and filtering of applications, assessment of applicants)
- intended to make decisions that affect, in particular, the termination and conditions of employment
are to be classified as high-risk AI systems. In this respect, the user or operator of AI systems, as well as providers, importers, dealers or manufacturers of AI systems, are subject to special obligations.
Works council’s co-determination rights
The employer is entitled to determine whether or not AI will be introduced in the employment relationship. However, the co-determination rights of the works council must be taken into account when determining the “how”. In particular, Art. 87 (1) No. 6 BetrVG (German Works Constitution Act) provides for a right of co-determination in the introduction and use of technical equipment that is objectively suitable for monitoring the behavior or performance of employees. If the works council has to assess the introduction and use of AI in order to perform its duties, it has the right to consult an expert. The employer must inform the works council about the planned use of AI as early as possible.
Discrimination
Using AI systems – just as human decisions – may entail discrimination. This is due to the fact that AI applications are fed by humans with training data that may already be unrepresentative and can therefore lead to unequal treatment. But how can this be countered legally? The German General Equal Treatment Act (“AGG”) also offers protection in this context, according to which employees may not be discriminated against either directly or indirectly on the basis of the discrimination criteria listed therein. Due to the technology-neutral wording of the AGG, this may also cover decisions made by AI applications. In the event of discrimination, the disadvantaged party is therefore entitled to compensation and damages under the AGG.
Requirement to perform the work in person
Employees are already using AI applications at work, such as DeepL for translating or Copilot / ChatGPT for composing texts. However, employees must perform their work personally, i.e., they are generally not allowed to delegate their work to third parties. It is therefore questionable whether the delegation of work tasks to an AI application constitutes the impermissible use of an auxiliary person or the permissible use of an auxiliary tool. The decisive factor here is likely to be that the work result is sufficiently checked by the employee for errors and is not passed off unchecked as their own work product. It is therefore advisable to draw up internal guidelines for dealing with AI applications and to train employees in this regard.
Outlook: Legal certainty as the key to a successful transformation
The AI Act’s coming into effect is to be welcomed and is also necessary in view of the risk of discrimination, as it is to be expected that the importance of AI in labor law practice will increase considerably. The interfaces between AI and labor law have already been the subject of court decisions.
The AI Regulation’s requirements affect all companies. It is to be expected that the topic of artificial intelligence will increase significantly in the coming years. This is due to several aspects: Most recently, the data protection organization Noyb filed a complaint against OpenAI claiming the generation of inaccurate information about individuals violated Art. 5 GDPR and the data subject’s right to rectification and erasure was not guaranteed.
The use of AI undoubtedly offers great opportunities for companies. Therefore, it is all the more worthwhile to implement it on a solid legal basis right from the start. We will be happy to advise you on all practical questions relating to data protection and compliance requirements.
Further information on the AI Act will be explained in more detail as part of the Data Protection Law Update series of events.