Your AI System Works. Can You Prove What It Did?

About the author

My name is Matteo Zanetti, and I serve as Chief Technology, HR and Finance Officer. Throughout my career, I have operated at the intersection of technology, finance, and organizational leadership, helping businesses transform innovation into sustainable growth and measurable results. This cross-functional perspective has given me direct insight into how artificial intelligence is transforming not only technology functions, but also decision-making, workforce management, and corporate strategy.

As AI rapidly evolves from a competitive advantage into a fundamental business capability, organizations face increasing expectations regarding governance, transparency, accountability, and regulatory compliance. Drawing on my experience leading digital transformation initiatives and evaluating AI adoption across multiple business domains, I wrote this article to provide executives and technology leaders with a practical framework for establishing effective AI governance in an increasingly complex regulatory environment.

Imagine a situation where a recruitment tool powered by AI suggests that a job candidate should be rejected. Now, let's say this decision is questioned six months down the line. The problem is, the company that made the tool has updated its model not once, but twice since then. The IT team can confirm that the system worked as it should when it made the recommendation, but they can't figure out which version of the model actually made the call or what exactly the human reviewer saw when they made their decision. This might sound like a made-up scenario, but it's actually a pretty believable mistake that can happen when engineering fails - a system can work just fine on the surface but not leave behind enough clues to help us understand how it arrived at a particular decision.

That gap is really important for Chief Technology Officers. Making sure AI systems follow the rules is not just about checking boxes on a list. It actually affects how things are built, which vendors are chosen, how different parts talk to each other, and what records are kept.

First, distinguish current duties from future ones

The new EU law on artificial intelligence, also known as the EU AI Act, affects companies that make and use AI systems, even if they are based outside the EU. This law has several parts that went into effect at different times. For example, the part about AI literacy, which is in Article 4, started on February 2, 2025. The rules about being transparent, which are in Article 50, began on August 2, 2026, but companies that already had certain AI systems got a little extra time to comply until December 2, 2026. Some important rules for high-risk AI systems, which are listed in Annex III, will start on December 2,2027. Then, on August 2, 2028, more rules will apply to specific AI systems that are used in products. It's worth noting that not all AI tools have to follow every single rule.

1. Know what is actually deployed

Let's start by keeping track of the tools we use inside the company, including features from vendors that are built into our systems and any AI tools that employees are using without permission, known as "shadow AI". We need to make a list that includes the supplier of each tool, who is responsible for it, what it's used for, what kind of data it handles, where it's being used, and whether it creates content or helps make important decisions. Before we start using any of these tools, we should figure out how they will be used and classify them. For example, systems used for hiring might fall under certain rules, like those outlined in Article 6, which has some exceptions.

A company needs to keep track of its internal registers, it's not something that has to be done just because of Article 4. The reason it's useful is that without it, the company won't know what's going on, so it can't provide the right training, figure out what the risks are, or even know which supplier to contact if something goes wrong. This is pretty important for the company to be able to respond quickly and effectively in case of an incident.

CTO decision: Who approves a new AI deployment, and where is its ownership and classification recorded?

2. Log for evidence, not just uptime

A successful API call is not a decision history. For sensitive workflows, consider records linking the transaction, relevant system/model information, input and output references, human intervention and final action. Keep personal data proportionate, secure and subject to defined retention periods. Test whether a disputed event can actually be reconstructed.

When it comes to high-risk systems, Article 26(6) says that the people in charge of these systems need to keep logs for at least six months. These logs are automatically generated and must be kept under their control, unless there's another law that says otherwise. On the other hand, Article 19 has its own set of rules for providers. It's worth noting that neither of these articles creates a strict six-month rule that applies to every single AI application out there.

3. Treat model changes as production changes

When a model is updated, the rest of the application and API can stay the same. But it's crucial to understand how these changes affect the system. So, we need to ask providers some key questions. Can they identify different versions of the model? How do they let us know about significant changes? And can they track the history of the output to see how it was configured? It's also important to test major upgrades and document any limitations we find. If a supplier can't give us version details, we need to be aware of that and plan accordingly. This way, we can make informed decisions and ensure the system works as expected.

California has a law called the AI Transparency Act, which was introduced by SB 942 and later changed by AB 853. This law shows why it's important to know where something comes from, especially when it comes to artificial intelligence. Some companies that use AI have to tell people about the AI they're using, including what kind it is and what version it is. They have to do this under certain conditions. Most of these rules started on August 2,2026, but some of the changes made by AB 853 will happen later. It's worth noting that not every company that uses chatbots has to follow these rules.

Can a supplier switch out the underlying model without the company realizing it's made a significant difference?

4. Keep AI-origin information with the content

When you create or edit an image, you can change it in many ways - like resizing it or sharing it again. But at what point do you lose the information about where the image originally came from? There's a rule, Article 50(2), that says some systems that generate images or other content have to add a special mark to the output. This mark is in a form that computers can read, so it's possible to tell if an image is real or fake. However, there are some exceptions to this rule. China has its own rules about labeling AI-generated content, which started on September 1,2025. These rules make a distinction between labels that you can see and metadata that's hidden behind the scenes.

Let's break down the process of handling content from start to finish. We need to figure out where to add information about the content's origin and test how it holds up when we make common changes to the files. It's not safe to assume that just one piece of information or one metadata field will make it through every change intact. Having a shared system to track the origin of our content can make it easier to work with people in other countries, but it's more of a guideline for designing our system rather than a strict rule that we have to follow.

CTO decision: Can the published asset still be identified under the processes and formats the company supports?

5. Make human oversight a real capability

Having an "Approve" button isn't really supervising if the person checking can't actually understand what's going on, question it, or say no to the suggestion. The rules say that companies using high-risk systems have to put someone in charge who knows what they're doing, has the right training, and has the power to make decisions. This is also connected to how the system is designed - it should be set up to help the person supervising understand what's happening, interpret the information, and be able to step in or stop the process if needed.

Let's take a closer look at the interface, permissions, and escalation path - all together. We should keep a record of any time a reviewer steps in, and what information they had access to, as long as it's legal and relevant. If needed, the engineering team can put in an override, but only if management gives the reviewer permission to use it. This way, we can make sure everything runs smoothly and that the right people have the right access.

CTO decision: Can the designated person genuinely depart from the model’s recommendation?

Procurement is where these controls begin

When working with sensitive workflows, it's crucial to ask vendors about certain key features before making any commitments. You'll want to know if they offer exportable logs, model identification, and change notices, as well as documentation and intervention controls. It's also important to consider provenance and incident support. While the law doesn't demand that every supplier keep every model or export every possible record, these capabilities can be vital for your intended deployment. So, make sure to assess them carefully and include them in the contract if they matter to you. This way, you can ensure that you have the necessary tools and support in place to meet your needs.

A Chief Technology Officer can keep an eye on four key things: how many AI systems have someone in charge and are labeled correctly, how many important workflows can be traced back to see how decisions were made, what percentage of critical suppliers have a plan in place for when models change and can show evidence, and how many workflows that need human review are actually tested. These numbers help turn ideas into real engineering tasks.

The point

When it comes to AI, there's no one-size-fits-all solution for making sure it complies with regulations. What's important is that these systems are easy to manage once they're up and running, not just impressive when you're showing them off. So, for a Chief Technology Officer, the question isn't just "Does the AI do its job?" but also "Can we figure out what it's doing, what's controlling it, and who can step in if needed?" Being able to do something is one thing, but being able to prove it and take responsibility is another. That's the difference between how well something works and being accountable for it.

Primary sources and editorial note

EU AI Act: Regulation (EU) 2024/1689, Articles 2, 4, 6, 14, 19, 26, 50 and Annex III — eur-lex.europa.eu/eli/reg/2024/1689/oj

EU amendment: Regulation (EU) 2026/1744 — eur-lex.europa.eu/eli/reg/2026/1744/oj

California: SB 942 and AB 853 — leginfo.legislature.ca.gov

China: Measures for Labelling AI-Generated and Synthesised Content (effective 1 September 2025) — cac.gov.cn

ThinkBeyondAI: “First a salary premium, now an EU law” — thinkbeyondai.com/ai-skills-premium-and-eu-law/

Here's how it works: some things like operational registers and audit-field choices are suggested by engineers, but they're not rules that everyone has to follow. What actually applies depends on the situation, what you're using it for, and where you are. This information is current as of October 16,2026.