Artificial Intelligence

How the BMW Group governs hundreds of agents across thousands of developers

When hundreds of agents serve thousands of developers, enterprise governance can’t be an afterthought.

At the BMW Group’s R&D, more than 12,500 developers work on approximately 500 million lines of vehicle software code. This software is augmented by the BMW Connected Vehicle backend, a cloud-based system of more than 1,300 microservices providing digital services for 24 million vehicles in the field. In regulated industry, the bar for (software) quality, stability, and governance compliance is exceptionally high.

While research, frameworks, and guidelines now facilitate building agents, industrializing and operating high-quality agents in an efficient, scalable manner remains a significant challenge.

Scaling agentic AI across a large organization cannot be achieved by isolated teams. It requires collective effort, locally driven experimentation, and active knowledge sharing across the enterprise. This creates a fundamental governance dichotomy: Development teams need the freedom to innovate rapidly, while the enterprise must enforce standards for safety, security, composability, and cost-efficiency. Without that balance, organizations risk a fragmented landscape of AI islands and costly duplicate work. This presents a real challenge given hundreds of globally distributed teams building agents across the BMW Group.

In this post, we detail the practices and patterns that support BMW’s R&D teams in building, governing and operating agents at scale without sacrificing speed, quality or compliance.

Solution

Our premise is straightforward: build agents that can cooperate from the start. This requires an architecture that makes diversity composable, governable, and automatable, without requiring a single framework or toolchain.

Central to this architecture is the agent harness, a runtime layer wrapped around a large language model (LLM). It provides the execution loop, tool dispatch, memory management, and observability hooks for autonomous operation.

Just as the AWS Well-Architected Framework standardized cloud infrastructure design, agentic systems need equivalent patterns, reusable blueprints encoding governance, composability, and operational best practices from the start.

We organize our solution around five interlocking practices, drawn from hands-on experience building and operating agentic systems at enterprise scale since 2023. Each practice is codified as infrastructure as code (IaC), making adoption repeatable, auditable, and enforceable across teams without blocking DevOps velocity.

Govern proactively, not retroactively

Governance must be structural from the start and maintained across the full lifecycle. We advocate for three properties of effective agentic governance:

Framework-agnostic and externalized: Governance rules such as authentication, authorization, rate limiting and content filtering are enforced at the integration layer. This includes a Model Context Protocol (MCP) gateway, a central agent registry and orchestration boundary, rather than within individual agent codebases. This means governance survives framework migrations, model swaps, and team reorganizations.

Domain-sensitive and graduated: Not every agent action carries the same risk: drafting documentation, updating customer records, and production database writes demand very different levels of oversight. We implement graduated trust through per-step autonomy levels:

  • Level 0 (Human-in-the-loop): Every action requires explicit human approval. This is the starting point for new agents and high-risk operations.
  • Level 1 (Human-on-the-loop): The agent executes autonomously but surfaces decisions for asynchronous human review. Suitable for medium-risk tasks with defined inputs, outputs, and rollback options.
  • Level 2 (Autonomous within guardrails): The agent operates independently within strictly defined policy boundaries, enforced deterministically rather than through prompt instructions (see below). Reserved for low-risk, well-understood operations with verified track records.

Assign trust levels per step, not per agent. A single workflow can include steps at all three levels, so teams can start conservatively and incrementally increase autonomy as confidence builds, grounded in measurable evaluation data, not assumptions.

Deterministically enforced: Governance runs as deterministic code outside the agent’s reasoning loop. In practice, this happens mainly in the CI/CD pipeline. Permission checks, policy-as-code rules, and compliance validations gate every deployment there, before an agent reaches production. Because these checks run separately from the agent, neither the agent nor its prompts are in a position to override them. Egress controls (recipient allowlisting, outbound secret scanning) help prevent an agent holding confidential data from disclosing it through an otherwise legitimate call.

Instrument, verify, and evaluate continuously

Observability in agentic systems goes beyond traditional application monitoring. The real question is no longer whether a service returned an HTTP 200 status, but whether an agent produced a faithful, policy-compliant output, and how this can be demonstrated through observable execution traces. Answering that requires a ground-truth dataset covering common queries and edge cases. This dataset is used for evaluation from day one, so quality is measured against representative and appropriately governed evaluation datasets for evaluation, not synthetic prompts.

This infrastructure cannot be retrofitted and must be part of the initial design. The continuous integration and continuous delivery (CI/CD) pipeline enforces this, so agents lacking instrumentation fail validation before evaluation runs.

We recommend instrumenting three dimensions:

  • Intent tracing: Capture the observable agent trajectory (reasoning steps, tool invocations, and context retrievals) for each execution. OpenTelemetry spans create a queryable lineage from user request to final output.
  • Verification and evaluation as CI: Any change to the agent (prompt edits, tool additions, model swaps) triggers the pipeline. Verification runs first with automated regression tests covering agent components and then their interactions. As most harness components are deterministic, these tests are fast and fully automatable. Evaluation runs second, scoring non-deterministic output against the ground-truth dataset, gated on quality thresholds. This sequencing is deliberate: Verification acts as a gate, so evaluations only run on a passing build.
  • Model tiering and routing telemetry: When agents route between models, capture the routing decision and its justification, not just the outcome. Tier assignments are derived from evaluation results per task type and poor outcomes flow back as new evaluation cases.

Start focused, design for composition

For distributed cloud systems, the industry solved the complexity of end-to-end workflows by adopting architectural patterns such as microservices or service meshes. The evolution of agentic systems needs to follow a similar trajectory, moving from monolithic, generalist agents toward multiple collaborating specialized agents.

Teams should define the problem before considering agents’ functionality. Before writing any orchestration logic, they should produce five concrete artifacts:

  1. Clear scope definition: what the agent should and shouldn’t do.
  2. Interaction model: how users and other agents will invoke it.
  3. Unambiguous tool definitions: each tool and parameter precisely defined.
  4. Ground-truth dataset: the evaluation reference described above, established before any implementation begins.
  5. Evaluation criteria: quality thresholds and benchmarks the agent will be measured against.

This discipline prevents the most common failure modes we observe: agents that take on too many responsibilities accumulate bloated prompts, reducing accuracy and increasing latency.

Once a single agent is well-scoped, it must be designed as a composable building block. Therefore, we define a three-tier composition hierarchy:

  • MCP tools are atomic integration contracts. Each tool wraps a single capability (for example, an API call or a database query) without encoding workflow assumptions. This is the stable, reusable layer.
  • Skills are bundled, domain-specific compositions of MCP tools. A code review skill, for example, might chain a static analysis tool, a style-check tool, and an LLM-based reasoning step. Skills encode domain logic but remain portable across workflows.
  • Flows are end-to-end orchestrations of skills, representing complete business processes (for example, an entire PR review pipeline or a requirements-to-deployment cycle).

Each tier is validated independently in the CI/CD pipeline. Component tests cover tools and skills, interaction tests cover their composition into flows.

These tiers are connected through open standards: MCP and Agent Skills for tool integration, the protocol for agent-to-agent communication, and OpenTelemetry for observability across the resulting interactions.

Figure 1 shows how the three tiers build on each other, from individual MCP tools at the base to complete flows at the top.

Figure 1: Three-tier system to build composable agents

Figure 1: Three-tier system to build composable agents

Specialize as you learn

In production, even well-scoped agents can break down as real-world usage diverges from what was anticipated at design time: query patterns shift, tool-selection logic struggles, and performance degrades unpredictably.

The right response is to decompose complex agents into multiple specialized ones, each with clearer instructions, simpler tool sets, and more focused reasoning. The benefits are concrete:

  • Evolve independently: updating a requirements engineering agent does not require touching the coding agent.
  • Scale individually: when scope and interfaces are well-defined, individual agents can be scaled independently without redeploying or resizing the broader system.
  • Minimized scope: a failure or hallucination of one agent does not cascade across the entire workflow.
  • Cost savings: A narrowly scoped agent can often run on a smaller model, which cuts inference and hosting costs.

Complexity can be reduced along two axes:

  • Within an agent: decompose capabilities into skills since tools and few-shot examples consume context on every invocation while degrading accuracy and increasing cost. Determine granularity empirically: merge skills the agent rarely selects and split skills that produce different behavior depending on the input.
  • Across agents: delegate work outside an agent’s domain to a specialized peer over the A2A protocol rather than extending the current agent. An agent should grow in capability, not in context.

Critically, since each specialized agent only accesses the data and tools required for its role, this architecture helps to enforce least-privilege by design, a compliance requirement in regulated industries.

Realizing these benefits, however, requires that agents can find and interoperate with each other. To prevent the emergence of AI islands, we maintain a centralized registry of agents, tools, and skills. This registry serves as:

  • Discovery mechanism: cataloging available agents, tools, and skills based on metadata stored during build.
  • Governance checkpoint: confirming approval of the asset types for production use and maintaining an audit trail.
  • Security checkpoint: automated security checks are applied before approval, such as prompt injection and similar issues.
  • Deduplication safeguard: identifying equivalent capabilities already maintained by other teams to avoid duplicate work. This check runs automatically at registration, flagging duplicates before approval.

Scale through codification

The shift from managing individual agents to governing an agent fleet requires agent architectures to be codified. Using Infrastructure-as-code frameworks such as Terraform or AWS CloudFormation, teams provision and manage agent infrastructure with the same rigor applied to any production system:

  • Fine-grained least-privilege roles: scoped per agent or tool.
  • CI/CD pipelines: consistent, validated deployments across accounts and regions, with automated deployment stops on failed tests or metric deterioration.
  • Policy-as-code: rules (for example, AWS Config, Open Policy Agent) scanning deployed agents for compliance with organizational standards.

All three are enforced as pipeline gates, validated before registration, not audited after the fact.

The adoption model is deliberately bottom-up with central curation. Individual teams build skills and MCP tools to solve their immediate problems. All teams can contribute by sharing their assets through the registry. A central team builds automatic governance checks as part of the CI pipeline to uphold standards. To support development teams and foster shift-left, the central team additionally provides building blocks as governance-as-code modules.

This balance avoids both centrally mandated architectures and unmanaged experimentation. As a result, when new operational concerns arise, teams can extend existing patterns rather than redesign their architectures.

Operationalizing best practices across the agent lifecycle

Architectural patterns and governance standards are only effective if they are embedded into the workflows teams use to build and operate agents before and at runtime.

Before deployment

Governance begins before a single line of orchestration logic is written. Teams are provided with reference architecture patterns and examples drawn from the registry. These pre-validated blueprints cover common agent configurations, tool integration patterns, and composition structures, which accelerate development while encoding proven defaults. Before anything reaches production, standardized CI/CD controls enforce these practices. Figure 2 shows the four phases of a reference pipeline, which run in order from left to right.

Figure 2: Reference pattern for a four-phase agent deployment pipeline.

Figure 2: Reference pattern for a four-phase agent deployment pipeline.

Phase 1 from Figure 2 runs fast, deterministic checks: supply chain security (approved registries only), component and interaction testing, evaluation library references, and observability instrumentation.

Phase 2 runs security and governance checks in parallel. Static analysis scans the code for issues, and security tests probe the agent in an isolated staging environment. The pipeline also validates egress controls and least-privilege permissions. Two more checks confirm that the agent’s declared autonomy level matches its actual permissions and that its configuration passes the policy-as-code rules. A governance agent accelerates audits by checking the agent against established standards and flagging findings for human review, such as:

  • Scope creep: operating beyond originally defined purpose or permissions.
  • Prompt bloat: unnecessary complexity in instructions.
  • Tool misuse: calling tools outside their intended use case.
  • Performance degradation: declining accuracy, latency, or cost-efficiency.

Figure 3: Example governance agent workflow

This assessment surfaces concrete, actionable measures that teams can apply immediately, rather than abstract recommendations. Figure 3 shows the governance agent’s dashboard for an agent maturity assessment. The governance agent provides evidence and actionable recommendations, while accountable humans own governance decisions. Recommendations require human confirmation before being applied, maintaining a four-eyes principle for accountability. Findings update the agent’s governance status in the registry, while full details are logged separately in a central knowledge base for audit and traceability.

Phase 3 evaluates the agent as a running system: the ground-truth evaluation gate (scores outputs against benchmarks established at design time) and agent performance (latency, token consumption). Figure 4 details the metrics, scoring criteria and the relationships between them.

Figure 4: Overview of evaluation metrics

Figure 4: Overview of evaluation metrics

Phase 4 checks for duplicates before registering the agent, its sub-agents, tools, and skills. The following table shows the metadata fields stored per agent in the registry.

Agent metadata
Core data:
  • unique identifier
  • name
  • description
  • version
  • business owner
  • IT owner
  • Capabilities: names, descriptions
  • Functions list: name, description, direction: provides/ uses
  • Status: plan / active / deprecated
  • Link to specification, agent card or documentation
  • Hosting mode: on-prem, cloud, cloud-region
Service level agreement data:
  • Target availability in percent
  • Target P95 response time in ms
  • Support hours / operational support model
Data governance flags:
  • information classification level
  • personal data processed flag
  • data retention period/ policy
  • request and response payloads logged flag
  • PII data redacted flag
  • Human in the loop required flag (required for high-impact / irreversible actions)

Table 1: Example agent metadata stored in the registry

If any step fails or metrics deteriorate, the deployment stops with logs for root cause analysis. If all checks pass, the agent is packaged and deployed as a containerized workload, producing consistent, reproducible deployments. This prevents ungoverned agents from reaching production, automates compliance checks, and keeps the registry accurate.

At runtime

At runtime, observability has to catch two kinds of drift. Data drift shows up as new or shifting user query patterns, or changes in upstream APIs. Model drift shows as rising error rates or falling tool-selection accuracy. When a customer flags a response, that also triggers an investigation. Alerts for drift, distribution shifts, declining evaluation scores, or direct customer flags prompt a staged response. These events are logged and forwarded to human review. Additionally, teams can run the governance agent continuously to flag drift and operational issues for the responsible team to investigate.

Summary

Building and operating agents at scale is an engineering discipline, not an experimentation exercise.

Effective governance must be structural from the start, externalized, domain-sensitive, and deterministically enforced. We introduced a three-tier composition hierarchy (MCP tools, skills, and flows) as the architectural foundation for reusable, composable, and interoperable agents, decomposed into specialized agents to contain complexity and scope. We showed how verification and evaluation in every CI/CD run make quality assurance part of everyday development, and how defining agent infrastructure as code lets teams apply the same governance rules to every agent in the fleet. Finally, we operationalized these patterns across the agent lifecycle, from pre-validated reference architectures and structured pipelines to a runtime governance agent that continuously evaluates deployed agents.

Our experience has shown these practices pay off in a few ways. Reusable templates and shared infrastructure reduced development costs. Teams also got agents to production faster, thanks to standard interaction models and automated deployment. Quality improved too, because the pipeline enforces proven patterns at every stage of the lifecycle.

Scaling agentic governance is an industry-wide imperative. The shift toward AI and generative-AI-supported workflow automation is already reshaping how engineering organizations operate. The winners will be those who move beyond experimentation and establish the discipline to govern autonomous agent networks at scale. With these practices in place, BMW engineers can focus on solving the problems that matter: Building the ultimate driving machine.

To apply these ideas to your own agents, start with the AWS Well-Architected Framework. This post used it as the model for reusable agent blueprints. To see how BMW codifies governance across its AWS environment, read How BMW Group uses automation to achieve end-to-end compliance at scale on AWS.

Outlook and next steps

The patterns and practices described in this post are a foundation, not a finish line. We are actively extending them across several fronts:

  • Improve architectural guidance: publishing best practice guides on architectural trade-offs, composition patterns, and anti-patterns for teams building agents.
  • Expand infrastructure-as-code templates: providing agent blueprints as usable IaC modules with built-in security and governance standards by default.
  • Deepen governance compliance testing: adding CI/CD test cases verifying governance compliance.
  • Automate standards enforcement: expanding automated checks and managed compliance-scanning rules across pre-deployment and runtime stages.
  • Extend agentic FinOps: Govern the right-sizing of models and tool calls.
  • Agentic user guidance: integrating described patterns into Archimedes, BMW’s architectural guidance agent, to provide contextual architectural recommendations.

Acknowledgements

The authors would like to thank (in alphabetical order) Alexander Stark, Alexander Zimmer, Dr. Céline Laurent-Winter, Dominik Segl, Florian Seidel, Hugo Lerias, Dr. Lars Lewerenz, Mark Neumann, Ralf Bottesch, Ramona Weber, Satish Vasireddy, Samuel Voss, Xagarey-Louis Meyer.


About the authors