Table of Contents
- Regulation Became Engineering Work
- Risk Tiers Determine Everything
- Are You a Provider or a Deployer?
- What High-Risk Obligations Mean in Code
- Transparency Obligations Are Broader
- The Documentation Nobody Wants to Write
- Building Compliance In
- Common Pitfalls
- Conclusion
- Frequently Asked Questions
Key takeaway: Compliance obligations are mostly requirements to document, test, log, and enable human oversight. Retrofitting these into a shipped system is dramatically harder than designing for them, because several require data you must have been collecting all along.
Regulation Became Engineering Work
For several years, discussion of AI regulation was speculative. It is no longer. Comprehensive frameworks now exist in multiple jurisdictions, with obligations that apply based on where your users are rather than where your company is registered.
The practical consequence for engineering teams is that a category of requirement previously handled entirely by legal now has technical dependencies. “Provide meaningful information about the logic involved” requires a system capable of explaining a decision. “Enable human oversight” requires an interface where a person can review and override. “Maintain logs” requires those logs to exist, with retention, from the beginning.
None of these are retrofittable cheaply. A system that did not log its inputs cannot produce historical records. A model with no recorded lineage cannot demonstrate what data trained it. This is why the practical advice is uniform: build for the obligations now, even if your current use case falls outside scope, because scope changes when your product does.
This article describes the shape of the requirements rather than legal specifics, which vary by jurisdiction and change. It is not legal advice, and anything with real consequences needs review by someone qualified in your jurisdiction.
Risk Tiers Determine Everything
The dominant regulatory pattern is risk-based tiering — obligations scale with potential for harm rather than applying uniformly.
Prohibited applications. Certain uses are banned outright: social scoring by public authorities, exploiting vulnerabilities of specific groups, and some biometric categorisation and untargeted facial image scraping. These are not compliance problems to solve; they are things not to build.
High-risk applications. The substantive obligations concentrate here. Typically includes systems used in employment decisions, education access, creditworthiness, essential services eligibility, law enforcement, and safety components of regulated products. If your system decides something significant about a person, assume this tier.
Limited-risk applications. Transparency obligations primarily. Chatbots must disclose they are not human. Synthetic media must be identifiable as generated. Emotion recognition must be disclosed to those subject to it.
Minimal-risk applications. Most software. Spam filtering, recommendations, inventory forecasting. No specific obligations beyond existing law.
The classification exercise is the first thing to do and it is frequently done wrong in a specific direction: teams classify by technical sophistication rather than by consequence. A simple logistic regression deciding loan eligibility is high-risk. An elaborate transformer suggesting playlists is minimal-risk. The regulation cares about what the decision does to someone, not how the system works internally.
Are You a Provider or a Deployer?
A distinction that determines which obligations attach to you, and one that confuses teams building on foundation models.
A provider develops a system and places it on the market. Providers carry the heavier obligations: conformity assessment, technical documentation, quality management, and registration where required.
A deployer uses a system in their own operations. Lighter obligations, focused on using the system as intended, ensuring human oversight, monitoring operation, and informing affected people.
The complication is that building on someone else’s model can make you a provider. If you take a general-purpose model and adapt it into a high-risk application — fine-tuning it, or building a system around it that makes employment decisions — you are likely the provider of that high-risk system even though you did not train the base model.
Teams routinely assume that using a vendor’s API leaves compliance with the vendor. It does not, for the system you built. The vendor is responsible for their model; you are responsible for your application of it. Contractual terms with the vendor matter for what information you can obtain, and they do not transfer your obligations.
What High-Risk Obligations Mean in Code
Translating the requirement categories into engineering work:
Risk management, continuous. A documented, maintained process identifying foreseeable risks and mitigations. Practically: a risk register that is updated when the system changes, not a document written once for an audit.
Data governance. Training data must be relevant, representative, and examined for bias. Practically: documented dataset provenance, demographic composition analysis, and recorded justification for inclusion and exclusion decisions.
Technical documentation. Enough detail for a third party to understand the design and evaluate compliance. Practically: architecture, training methodology, evaluation results disaggregated by group, known limitations.
Automatic logging. Records of operation sufficient to trace decisions. Practically: input, output, model version, timestamp, and any human override, retained for a defined period. This is the obligation most often impossible to satisfy retroactively.
Transparency to deployers. Instructions covering intended use, limitations, expected accuracy, and required oversight. Practically: real documentation, not a marketing page.
Human oversight. Design permitting effective human intervention. Practically: an interface showing the decision and its basis, with an override that works and is recorded.
Accuracy, robustness, and security. Documented performance, resilience to errors and adversarial input, and appropriate security. Practically: your evaluation suite, adversarial testing, and access controls.
Reading that list, most of it is what a well-engineered system should have anyway. The difference regulation makes is that “should” becomes “must”, and undocumented good practice does not count.
Transparency Obligations Are Broader
The transparency tier catches more systems than teams expect, because it applies based on interaction pattern rather than consequence.
Disclosure of AI interaction. If a person interacts with an AI system, they must know. This applies to chatbots and voice agents regardless of what they discuss. The bar is that a reasonable person would understand, which means a disclosure buried in terms of service is insufficient.
Marking synthetic content. Generated or manipulated image, audio, video, and text must be machine-readable as such. Practically: content credentials metadata and watermarking where feasible.
Emotion recognition and biometric categorisation disclosure. People subject to these must be informed. Some contexts, notably workplace and education, face stricter limits or prohibition.
Deepfake labelling. Content depicting real people saying or doing things they did not requires clear disclosure.
The synthetic content marking obligation deserves particular attention because it applies to products that do not otherwise think of themselves as regulated. A marketing tool generating product images, a writing assistant producing article drafts, an audio tool cloning a voice for accessibility — all fall within it.
The Documentation Nobody Wants to Write
The single most valuable compliance artefact is a maintained system card. It satisfies several obligations simultaneously and is genuinely useful independent of regulation.
Contents that matter:
Purpose and intended use. Specifically what decisions this system informs and in what context.
Out-of-scope uses. Where it must not be used. This section prevents more harm than any other and is the one most often omitted.
Data provenance. Sources, time period, size, demographic composition, known gaps, and how it was collected and licensed.
Performance, disaggregated. Accuracy and error rates overall and by relevant group. Aggregate-only reporting does not satisfy scrutiny.
Known limitations. Where the system performs worse, which inputs it handles poorly, what it cannot do.
Human oversight design. How a person reviews decisions, what information they see, how override works.
Change history. Model versions, when deployed, what changed, and what was re-evaluated.
That last item is what turns documentation from a snapshot into a record. A system card describing the version deployed eighteen months ago is worse than none, because it is confidently wrong.
Building Compliance In
An ordering that avoids expensive retrofitting:
Log from the first deployment. Input, output, model version, timestamp, human override. Retention policy defined upfront. This is cheap now and impossible later.
Version the four training inputs. Code, data, configuration, environment — linked by one identifier. Data provenance obligations are unsatisfiable without this.
Evaluate disaggregated by default. Build group-level metrics into your evaluation suite from the start. Adding them later means re-running historical evaluations you may not be able to reproduce.
Design the override path early. Human oversight is an interface requirement, not a policy statement. Retrofitting it into an automated pipeline is a substantial change.
Write the system card while building. Written during development it is accurate and cheap. Written for an audit it is archaeology.
Classify before designing. Knowing your risk tier changes architectural decisions. Discovering it after launch changes them expensively.
Retain evaluation results. Not just current metrics — the history. Demonstrating that you tested before each deployment requires the records to exist.
Common Pitfalls
Assuming the model vendor’s compliance covers you. They are responsible for their model, you for your system. Building a high-risk application on a compliant model does not make your application compliant.
Classifying by technical complexity. Consequence determines the tier. Simple models making consequential decisions are high-risk.
Treating this as legal’s problem. Logging, versioning, disaggregated evaluation, and override interfaces are engineering deliverables.
Documenting once. Obligations are continuous. A system card describing a superseded version is actively misleading.
Insufficient logging. The most common unsatisfiable obligation, because historical records cannot be created after the fact.
Ignoring transparency because you are not high-risk. Synthetic content marking and AI disclosure apply broadly.
Aggregate-only performance reporting. Disaggregation by group is expected and reveals what totals hide.
Conclusion
AI regulation has moved from anticipated to operational, and most of what it requires is engineering work: logging, versioning, disaggregated evaluation, documented limitations, and interfaces enabling genuine human oversight.
The practical strategy is to build these regardless of your current classification. Log operations from day one, because those records cannot be generated retroactively. Version training inputs together, because provenance obligations depend on it. Evaluate by group by default. Design the override path as a feature rather than a policy. And maintain the system card as you build, when it is accurate and cheap.
Teams that do this find compliance mostly a matter of assembling what they already have. Teams that do not find it a project, conducted under deadline, on systems that were not designed to produce the required evidence.
Frequently Asked Questions
Does this apply to companies outside the regulating jurisdiction? Generally yes, where your system is used by or affects people within it. Territorial scope typically follows the users rather than the company’s registration.
We only use a vendor API — are we exempt? No. You are responsible for the system you built on it. Depending on how you adapt and deploy it, you may be a provider of a high-risk system regardless of not having trained the model.
How is a high-risk classification determined? By application domain and consequence — employment, credit, education, essential services, law enforcement, safety components. Not by model architecture or sophistication.
What logging is actually required? Enough to trace how a decision was reached: input, output, model version, timestamp, and any human intervention. Retention periods vary by framework and are typically measured in years for high-risk systems.
Is a system card sufficient documentation? It covers a meaningful portion of the technical documentation obligation and is not the whole of it. Risk management records, data governance documentation, and evaluation history are separate requirements.
What about open-source models? Frameworks generally treat research and free open-source release more leniently than commercial deployment, with exceptions where the model is used in high-risk applications. Releasing a model and deploying it commercially attract different obligations.
How much does compliance cost? Considerably less if built in than retrofitted. The expensive scenario is discovering that a deployed system cannot produce required evidence, which means adding instrumentation and re-running evaluations under time pressure.
