The EU AI Act Takes Effect in August. Here's What Your AI Infrastructure Needs to Do.

April 15, 2026 17 min read

EU AI Act compliance governance proxy audit-logging enterprise regulation

The EU AI Act’s high-risk provisions take effect August 2, 2026. Penalties hit 35M EUR or 7% of global turnover. Here is what compliance looks like at the infrastructure layer, not the application layer.

The number that matters

35 million euros, or 7% of global annual turnover, whichever is higher. That is the maximum penalty under Article 99 of the EU AI Act for violations related to high-risk AI systems. For context, 7% of turnover makes GDPR’s 4% look like a warm-up.

On August 2, 2026, Annex III’s high-risk provisions become enforceable. If you deploy AI systems in areas like employment, creditworthiness, critical infrastructure, or law enforcement, your systems must comply with a set of technical requirements that go far beyond “write a policy document and file it somewhere.”

The requirements are specific. They demand documented risk management (Article 9), training data governance (Article 10), technical documentation (Article 11), automatic logging (Article 12), transparency obligations (Article 13), and human oversight mechanisms (Article 14). Each of these has infrastructure implications. Most organizations are not ready.

According to Deloitte’s 2026 State of AI report, only 21% of organizations deploying agentic AI have mature governance models in place. Over half lack even a systematic inventory of what AI systems they have running in production. Meanwhile, 84% plan to increase AI investment this year. The gap between ambition and compliance infrastructure is growing, not shrinking.

Why application-level compliance fails

The instinct is to solve compliance at the application layer. Add logging to the agent code. Build a risk assessment into the prompt template. Create a dashboard that someone in compliance can check once a quarter.

This approach has three problems.

Coverage. If you run 30 agents across five teams, application-level compliance requires every team to implement every requirement in every agent. Miss one, and you have a gap. The EU AI Act does not care which team forgot to add logging.

Consistency. Different teams implement the same requirement differently. One team logs to stdout. Another writes JSON to S3. A third logs request metadata but strips the response body. When the auditor asks “show me your automatic logging system,” you show them three incompatible approaches and hope for the best.

Enforceability. Application-level controls live inside the process they are supposed to govern. An agent can skip its own logging. A developer can disable the risk check in a hotfix. There is no enforcement boundary between the system and the control. This is the same architectural problem that made the Replit database deletion possible: when the guardrail lives inside the thing it guards, the thing can remove the guardrail.

Infrastructure-layer compliance solves all three. One system, all traffic, consistent enforcement, no opt-out.

Article by article: what the infrastructure must do

Here is how the key technical requirements in the EU AI Act map to infrastructure capabilities.

Let us walk through each one.

Article 9: Risk management through policy enforcement

Article 9 requires a “risk management system” that identifies, analyzes, and mitigates risks throughout the AI system’s lifecycle. This is not a one-time risk assessment. It is a continuous process.

At the infrastructure layer, this translates to a policy engine that evaluates every AI request against predefined risk criteria before it reaches the LLM provider.

policies:
  risk_management:
    - name: high_risk_content_filter
      condition:
        prompt_contains_any: ["personal data", "credit decision", "employment"]
      action: flag_for_review
      risk_category: "annex_iii_high_risk"

    - name: cost_anomaly_detection
      condition:
        request_cost_exceeds: 5.00
      action: require_approval

    - name: model_restriction
      condition:
        risk_category: "annex_iii_high_risk"
      allowed_models: ["gpt-4o", "claude-sonnet-4-6"]
      block_models: ["gpt-4o-mini", "gpt-3.5-turbo"]

Three policies. High-risk content gets flagged. Cost anomalies require human approval. High-risk categories are restricted to models with documented capability assessments. None of this requires changes to agent code. The agents send requests. The proxy enforces policy through YAML configuration.

The critical distinction: this is not a suggestion to the agent. It is a network-layer gate. The request does not reach the LLM provider until the policy engine passes it. The agent cannot skip the check, disable it, or route around it.

Article 12: Automatic logging that actually works

Article 12 is specific about what “automatic logging” means. The logging must capture events “relevant to identify risk,” must be proportionate to the intended purpose of the system, and must enable monitoring of the system’s operation. The logs must be retained for an “appropriate” period.

This is not console.log(request). This is a structured, immutable, queryable audit trail that captures every AI interaction in your organization.

At the proxy layer, every request and response passes through a single point. Logging is not optional and not delegated to individual agents. The proxy captures:

Timestamp, agent identity, organization
Full request payload (prompt, model, parameters)
Full response payload (completion, token usage, latency)
Policy evaluation results (which policies were checked, which triggered)
Risk classification of the request
Whether human oversight was invoked
Final disposition (forwarded, blocked, modified)

This is the same logging architecture described in detecting token count manipulation, extended with compliance-specific fields. The proxy already logs every request for cost integrity. Adding risk classification, policy evaluation results, and human oversight flags is incremental, not architectural.

When an auditor asks “show me every AI decision your system made in the last 90 days for high-risk use cases,” you run a query. You do not ask five teams to grep their application logs.

Article 14: Human oversight that is not theater

Article 14 requires that high-risk AI systems be “designed and developed in such a way that they can be effectively overseen by natural persons.” The humans must be able to “fully understand the capacities and limitations” of the system and to “intervene on the functioning of the high-risk AI system or interrupt the system.”

In practice, most organizations implement human oversight as a dashboard someone can look at. The AI makes decisions. A person can, theoretically, review them. Nobody does, because the volume is too high and the dashboard updates too slowly.

Real human oversight requires intervention hooks at the infrastructure layer:

oversight:
  high_risk_agents:
    - name: credit_decision_agent
      require_human_approval:
        when: "risk_category == 'credit_scoring'"
        timeout: 300  # seconds
        escalation: block_request
        notify: ["compliance-team@company.com"]

    - name: employment_screening_agent
      require_human_approval:
        when: "risk_category == 'employment'"
        timeout: 600
        escalation: block_request
        notify: ["hr-compliance@company.com"]

  kill_switch:
    enabled: true
    scope: per_agent
    activation: api_call  # POST /api/agents/{id}/disable

The proxy holds the request. It notifies the designated human. If the human does not respond within the timeout, the request is blocked, not forwarded. The kill switch disables an agent entirely via API call.

This is not oversight theater. It is a circuit breaker in the request path. The human is not reviewing a dashboard after the fact. They are in the loop before the AI output reaches the downstream system.

Article 11: Documentation that writes itself

Article 11 requires technical documentation that includes the system’s intended purpose, how it was designed, and how it was tested. For most teams, this means a painful documentation sprint before every audit.

Proxy-based governance inverts this. The policy configuration is the documentation.

# This YAML file IS the technical documentation for Article 11
system:
  name: "Customer Credit Assessment Pipeline"
  purpose: "Automated preliminary credit scoring for consumer applications"
  risk_category: "annex_iii_high_risk"
  eu_ai_act_classification: "Article 6(2), Annex III, point 5(b)"

agents:
  credit_scorer:
    allowed_models: ["gpt-4o"]
    max_tokens_per_request: 4000
    policies: [pii_filter, bias_check, human_approval_required]
    logging: full
    human_oversight: required
    data_governance:
      input_validation: strict
      pii_handling: redact_before_inference
      output_validation: structured_schema

This configuration is version-controlled. Every change is a git commit with a timestamp and author. The history of your governance decisions is the git log. When the auditor asks “when did you add PII filtering to the credit scoring pipeline?” you do not need a meeting. You have a commit hash.

The proxy vs SDK architecture article covers why this centralized configuration model is more maintainable than distributing governance logic across individual agent codebases. For compliance purposes, the advantage is even sharper: one place to audit, one place to update, one place to prove conformity.

The inventory problem

Before you can comply with any of these requirements, you need to know what AI systems you have. Article 9 requires risk management across the “entire lifecycle.” You cannot manage the risk of systems you do not know exist.

Over half of organizations lack a systematic inventory of AI systems in production. Teams spin up agents, connect them to LLM providers, and deploy them without telling anyone. The AI equivalent of shadow IT.

A proxy architecture solves this by design. If every LLM call must go through the proxy, the proxy is the inventory. Every agent that calls an LLM is registered. Every model it uses is recorded. Every request is logged. There is no shadow AI because there is no way to reach the LLM provider without going through the proxy.

This is the same principle behind eliminating shared secrets in multi-tenant infrastructure. When the agent never holds the real API key, the proxy is the only path to the provider. That architectural constraint, designed for security, turns out to be the foundation of compliance.

What to do before August 2

If you deploy AI systems that fall under Annex III’s high-risk categories, here is the minimum viable compliance path at the infrastructure layer.

1. Inventory your AI systems. Route all LLM traffic through a proxy. If an agent cannot reach the provider without going through the proxy, the proxy is your inventory. No spreadsheet required.

2. Classify by risk. Tag each agent with its risk category in the proxy configuration. High-risk systems get stricter policies, mandatory logging, and human oversight hooks.

3. Enable automatic logging. Full request/response capture with policy evaluation results, risk classification, and oversight decisions. Immutable, append-only, queryable. Retention period aligned with your sector’s requirements.

4. Implement human oversight. Not a dashboard. Intervention hooks in the request path. Approval workflows for high-risk decisions. Kill switches per agent. Timeouts that default to blocking.

5. Version-control your policies. YAML configuration in git. Every policy change is a commit. The git log is your documentation trail for Article 11.

6. Test the audit path. Simulate an auditor’s questions. “Show me every credit decision your AI made last quarter.” “When was human oversight added to this agent?” “What changed in your risk management policies in March?” If you cannot answer these from your infrastructure, you are not ready.

Check your readiness now

The interactive EU AI Act Readiness Checker maps all 24 infrastructure requirements from Articles 9 through 14. Check off what you have in place and see your compliance score, gap analysis, and days remaining until enforcement.

Open the EU AI Act Readiness Checker →

The compliance advantage of proxy architecture

The EU AI Act’s requirements are not arbitrary. They describe what responsible AI operations look like: know what you are running, log what it does, manage the risks, let humans intervene, document everything.

A proxy architecture satisfies multiple Articles through a single infrastructure choice. Not because it was designed for the EU AI Act. Because the same properties that make a proxy good for security, cost management, and audit integrity are the properties the regulation demands: centralized visibility, consistent enforcement, immutable logging, and a clear boundary between the AI system and the controls that govern it.

August 2 is 114 days away. The regulation is final. The penalties are real. The question is not whether your AI systems need governance. It is whether the governance lives at the right layer.