02/01/2024 -Recently, the Court of Justice of the European Union (hereinafter: the Court) has made an important ruling on the relationship between ‘credit scoring’ and automated decision-making, in the context of the GDPR. The implications of the ruling are not only limited to credit scoring but may also extend to the preparation of decisions by AI applications.
Broader definition of automated decision-making
The importance of this ruling lies in the fact that it introduces a broader interpretation of automated decision-making. The case (case C-634/21) involved the German credit bureau SCHUFA, which provides information about individuals to credit providers. Credit providers then used this information to determine whether customers could or could not get a loan.
The Court explains that not only a decision itself but also the preparatory actions for that decision can count as an automated decision. However, the condition is that the preparatory actions are a significant factor in the final decision. This ruling thus emphasizes that the responsibility for complying with Article 22 of the GDPR, which is about the rights of individuals in automated decision-making, lies not only with the credit provider but also with the information provider.
Elements of Automated Decision Making
The Court listed the elements that are necessary to speak of automated decision-making:
- First, an actual decision must be made;
- Second, this decision must be based solely on automated processing, including profiling;
- Third, the decision must have legal effects concerning the individual or otherwise produce an equivalent or similarly significant effect in its impact on the individual.
The Court explained that 'decision' is a broad concept and that various actions can have an impact on individuals, including credit scoring. Furthermore, it was remarkable that the Court pointed out that the preparatory actions significantly impact an individual because there is a strong reliance on automated processing.
Implications for AI-based Decision Making
Due to the broad interpretation of 'automated decision making,' the judgment is also particularly relevant in the context of AI applications, as these are often used in the preparation of decisions. Thus, parties that prepare information using AI may also be subject to the restrictions and requirements of Article 22.
An example is the use of an AI system that assesses the suitability of candidates during the first phase of a recruitment process. The preparatory actions of the AI system and presenting, for example, a 'suitability score' can be seen as automated decision-making if there is a heavy reliance on this information when choosing an applicant.
Another example is the use of AI in healthcare to recommend treatments and medication. These AI-generated recommendations can then be used as a significant part of the diagnostic or treatment decisions.
Evaluation of AI Applications
The Court's ruling represents a step in shaping the future interaction between technology, privacy, and legislation. It requires a careful evaluation of how AI systems are deployed, with particular attention to transparency, consent, and the rights of individuals to challenge decisions.
At Considerati, we understand the importance of these developments and help organizations ensure compliance and implement best practices. Should you have any further questions regarding the use of profiling, automated decision-making, and the requirements set forth under the GDPR, please contact us.