This piece first appeared in German. The English version is a rewrite rather than a line-by-line translation, and the German original stays online in the archive: KI trifft auf Regulierung - Beispiel: Banken.

Imagine you apply for a loan. Steady income, no outstanding debt, nothing unusual in the file. The rejection arrives anyway, automatically, within seconds. No case handler, no reasons given, no one to ask. Somewhere in a data center an algorithm decided you were not a suitable borrower.

What produced that decision stays out of sight. It might have been the postcode. Or a feature in the training data that has nothing to do with creditworthiness and correlates with it statistically. BaFin, the German financial supervisor, describes exactly this risk. A training dataset can leave out particular customer groups, which makes it unrepresentative of the actual customer base, and the algorithm then assesses those people without an empirical basis. Such a model discriminates because the data reflects reality unevenly, and not because anyone programmed it to.

That is the regulatory problem in one line. Highly automated decision processes with little human supervision can amplify discrimination risks that already exist, as BaFin puts it. It is why the European legislator wrote the AI Act. Banks using AI in lending or risk management now carry concrete duties, and the European legislator has already moved the date on which they apply once.

Credit scoring is named as high-risk in the law itself

Regulation (EU) 2024/1689 entered into force on 1 August 2024 and takes effect in stages. One date matters most for banks, and it has moved. The full obligations for high-risk AI systems were due on 2 August 2026, and the Digital Omnibus on AI, Regulation (EU) 2026/1744, postponed them for systems like credit scoring to 2 December 2027.

The law works from a risk-based approach, so the more deeply a system can reach into fundamental rights, the stricter the requirements on it. For banks the high-risk class is the one that counts, because a core part of the business sits inside it.

Annex III lists the applications that carry the classification. The entry for the financial sector is point 5(b): AI systems intended to evaluate the creditworthiness of natural persons or establish their credit score. Systems used to detect financial fraud are excluded. The Bankenverband, the association of German banks, welcomes that distinction and asks the Commission for further clarification on anti-money-laundering and sanctions screening.

High-risk does not mean dangerous. In the language of the AI Act it means a system that shapes decisions with a serious effect on individuals. Credit scoring fits the definition because it governs access to financial services.

The duties are specific, and none of them is a formality

Anyone providing or operating a high-risk AI system has to meet several requirements. A continuous risk management system, under Article 9. Demonstrated data quality, including bias testing of the training data, under Article 10. Complete technical documentation, under Article 11, and mechanisms for human oversight.

Article 6(3) does allow an exemption from the classification where a system has no material influence on the outcome of a decision. The idea that an AI pre-check in lending escapes the classification because a person signs off is not supported by the regulation, as TÜV Consulting reads it. An upstream scoring model that hands a case handler a recommended value can fall under the high-risk requirements too.

In its position paper of July 2025 the Bankenverband asks for a clearer separation of roles between providers and deployers. That matters most where banks share AI inside a group structure or work with external service providers. Uncertainty about who holds which role, the association argues, damages legal certainty and can slow progress down.

Fairness was already law before the AI Act arrived

A second strand of regulation deals with discrimination, and it sits in existing law independently of the AI Act. BaFin separates two forms. Direct discrimination puts a person at a disadvantage explicitly because of a protected characteristic such as age or gender. Indirect discrimination happens when apparently neutral criteria, income level for instance, systematically disadvantage particular groups.

Both forms are legally problematic without being absolutely prohibited. Unequal treatment that can be justified on the facts, because the criterion demonstrably correlates with actual default risk, counts as permissible. Where legitimate risk differentiation ends and unlawful discrimination begins is one of the central questions in any AI system that touches lending.

The risk usually sits in the data. In a machine learning context BaFin defines bias as a systematic distortion of results, arising for example when training data does not represent particular groups adequately. Feed a model historical lending decisions and past structural disadvantage travels into it, where it can take on a life of its own.

Article 10 of the AI Act sharpens that expectation into a condition: a high-risk system has to be trained on demonstrably representative data and tested for bias before it goes into operation.

Several rulebooks apply to the same system at once

The regulatory picture for AI in banking is complicated because more than one framework is live. In Germany for examlple, there are four of them: the AI Act, the GDPR, the MiFID II regime, and MaRisk, the German supervisory rules on risk management in banks. Germany has since added KI-MIG, the national act implementing the AI Act, which the federal cabinet adopted in February 2026.

The Bankenverband argues for consistent application of the AI Act across the Union, so that supervisory practice is harmonized and every market participant works under the same conditions. Fragmented national implementation would produce the opposite: different compliance requirements depending on location, and a disadvantage against competitors from outside the EU.

The problem was never the law, it was the lead time

A risk-based framework makes sense for a sector with this much effect on people's lives, and what is hard about it is the speed of implementation. The postponement gives the many institutions that have tested AI mainly in pilot projects sixteen more months, and it does not change what they have to build.

Regulation and innovation do not exclude each other in finance, but they do require lead time. Banks treating AI governance as a strategic task can turn the requirements of the AI Act into an advantage. Those still waiting have until 2 December 2027, which is shorter than it sounds for a model that has to be documented, tested for bias and placed under human oversight.

Sources

Grouped by the section they support.

Opening

  • BaFin (2024), KI (Künstliche Intelligenz) bei Banken und Versicherern: Automatisch fair?, BaFinJournal, 1 August 2024, by Lydia Albers, Dr. Matthias Fahrenwaldt, Ulrike Kuhn-Stojic and Dr. Martina Schneider, bafin.de. Supports training data that leaves out customer groups, and highly automated decisions with little human supervision amplifying existing discrimination risks. The loan applicant is an illustration, not a reported case.

Credit scoring is named as high-risk in the law itself

  • Regulation (EU) 2024/1689 of 13 June 2024 (AI Act), Official Journal, 12 July 2024, eur-lex.europa.eu. Supports the entry into force (Article 113), the risk-based approach, and Annex III point 5(b), including the exception for fraud detection. Article texts via the Commission's AI Act Service Desk.
  • Regulation (EU) 2026/1744 of 8 July 2026 (Digital Omnibus on AI), Official Journal, 24 July 2026, eur-lex.europa.eu. Moves the application date for high-risk systems under Annex III from 2 August 2026 to 2 December 2027.
  • Bundesverband deutscher Banken (2025), Positionspapier zu einem KI-förderlichen Rechtsrahmen, 7 July 2025, bankenverband.de. Supports the association's welcome of the fraud exception and its request for clarification on anti-money-laundering and sanctions screening.

The duties are specific, and none of them is a formality

  • AI Act, Articles 9 (risk management system), 10 (data and data governance), 11 (technical documentation) and 14 (human oversight), and Article 6(3), including the rule that an Annex III system that profiles natural persons is always high-risk, AI Act Service Desk.
  • TÜV Consulting (2026), Hochrisiko-KI nach Anhang III: Welche Systeme der EU AI Act erfasst, by Cathrin Ribbrock, 23 April 2026, consulting.tuv.com. Supports the reading that a human final decision does not remove an upstream scoring model from the high-risk class, because classification follows the system's influence on the decision.
  • Bundesverband deutscher Banken (2025). Supports the call for a clearer separation of roles between providers and deployers, including group structures.

Fairness was already law before the AI Act arrived

  • BaFin (2024). Supports direct and indirect discrimination, justified unequal treatment, and the definition of bias as a systematic distortion of results.
  • AI Act, Article 10, AI Act Service Desk. Supports the examination for possible biases (paragraph 2(f)) and sufficiently representative data (paragraph 3). The point about historical lending data is the author's own.

Several rulebooks apply to the same system at once

  • Regulation (EU) 2016/679 (GDPR), eur-lex.europa.eu. Directive 2014/65/EU (MiFID II), eur-lex.europa.eu. BaFin, Rundschreiben 06/2024 (BA): Mindestanforderungen an das Risikomanagement (MaRisk), 29 May 2024, bafin.de. The three further frameworks named in the text.
  • Federal Ministry for Digital Transformation and Government Modernisation (2026), Kabinett beschließt schlanke KI-Aufsicht in Deutschland, press release, 11 February 2026, bmds.bund.de. Supports the cabinet decision on the KI-Marktüberwachungs- und Innovationsförderungsgesetz (KI-MIG)
  • Bundesverband deutscher Banken (2025). Supports the call for harmonized supervision across the Union and the warning against national fragmentation.

The problem was never the law, it was the lead time

  • Regulation (EU) 2026/1744. Supports the current application date for high-risk systems. The assessment of lead time and governance is the author's own.
The link has been copied!