Who is making the (automated) decisions?

In December last year, the CJEU handed down a landmark judgment in the case of Schufa (Case C634/21) regarding the interpretation of “automated individual decision-making” under Article 22 GDPR.

Specifically the CJEU held that Schufa, a German credit reference agency, engages in ADM under Article 22 when it generates credit scores which a third-party lender then “draws strongly” on when deciding whether to lend to individuals. The CJEU also held that Article 22(1) should be interpreted as a prohibition in principle on using ADM, i.e. it does not have to be invoked by an individual.

The decision attracted significant attention because the CJEU’s interpretation of “decision-making” extended the scope of Article 22 to service providers generating a score or probability value. In Schufa, the lender (not the credit reference agency) decided whether to offer the individual a loan and on what terms. The CJEU held, however, that Schufa was subject to Article 22 because the lender’s decision drew strongly on the credit score it generated. The Court held that the term “decision” could encompass “a number of acts which may affect the data subject in many ways”, including calculating a credit score.

In principle, the case means that a service provider whose automated processing services are drawn strongly upon by a third party to “establish, implement or terminate a contractual relationship” with an individual could be caught by Article 22. In Schufa, the CJEU held that the credit score played a “determining role” in the decision, noting that a low credit score would “in almost all cases” result in the bank rejecting a loan application (as happened to the individual who brought the case). Therefore, if a third party relies heavily on other factors when party relies heavily on other factors when making a decision about an individual, then the service provider’s processing may not reach the threshold for ADM. It is hoped that data protection authorities will provide further guidance on what “draws strongly” means in practice in the context of Article 22.

Automated decision-making, particularly based on AI, is prevalent, most notably in sectors such as insurance, finance recruitment, and healthcare, where automated scoring or evaluation metrics may be used. The Schufa decision will have significant implications for organisations using such technology. It underlines the importance of developing an AI governance framework to ensure compliance with the GDPR and the proposed AI Act – and illustrates that both service providers and their customers need to be aware of their legal obligations concerning ADM.

In the UK, the Data Protection and Digital Information Bill (currently being debated in Parliament) proposes to remove the general prohibition on ADM from Article 22 of the UK GDPR, such that the prohibition would only apply where special category personal data is processed. The Bill still requires controllers to implement safeguards for other uses of ADM. If the Bill is passed in its current form, Schufa may have even less impact in the UK. Although CJEU judgments handed down post-Brexit do not bind UK courts, the ICO has continued to refer to EU enforcement action and case law where relevant to its investigations or decisions.

This article is part of our Data Protection Top 10 2024 publication.