B6. Operating Model and Organization
While the architecture defines which technical building blocks are needed, the operating model answers who develops, operates, and is accountable for them, and whether AI is steered centrally or distributed across business units. The input comes from chapter B1: there, executive management assigned overall accountability; here, the organization beneath it is designed. Without a clear operating model, uncoordinated initiatives, duplicated effort, and a lack of scalability follow. The goal is a repeatable, scalable, and secure approach across the entire lifecycle, from idea to decommissioning.
Scope: this chapter defines the organizational form and the operating processes. The concrete roles, decision rights, and templates (RACI, role profile) are bundled in chapter B7.
What this chapter delivers: the operating model along six questions on three levels. First the organization itself, then the path a single initiative takes through it, finally the rules that apply to all of them and the steering that keeps the operating model effective. The first column states at any point which level a section sits on:
| Level | Section | Question |
|---|---|---|
| The organization | Organizational form | How are decision rights distributed: centralized, federated, or hybrid? |
| The organization | AI competence center | What does the central team concretely deliver, how is it staffed and funded? |
| The path of an initiative | Intake | How do ideas enter the portfolio in an orderly way instead of by acclamation? |
| The path of an initiative | Lifecycle management | Which approvals does an AI model need from idea to decommissioning? |
| Rules and steering | What is binding for every initiative | Which working practices are mandatory, who oversees them, how are partners steered? |
| Rules and steering | Steering and evolving the operating model | How does the operating model measure itself and stay effective? |
Organizational form: centralized, federated, or hybrid
The organizational form determines how decision rights and competencies are distributed. This is the central structural decision of the operating model.
| Organizational form | How it works | Strengths | Risks | Countermeasures |
|---|---|---|---|---|
| Centralized (AI competence center) | A central team pools expertise, provides the platform, sets standards | Consistency, high reuse, fast scaling | Bottleneck with many projects | Service catalog, transparent prioritization, capacity steering |
| Federated (with guardrails) | Business units develop their own products on shared building blocks | Closeness to the business, speed, innovation | Inconsistency, duplicated effort, architecture drift | Hard guardrails, mandatory reviews, communities of practice |
| Hybrid | Central guardrails and building blocks, decentralized delivery | Balance of consistency and flexibility | Unclear central/decentral boundaries | Clear mandates, documented interfaces |
Selection guide: which organizational form fits when
The table matches typical starting situations to an organizational form; the rationale names the decisive factor in each case:
| Situation | Recommendation | Rationale |
|---|---|---|
| Entry phase, few experts, first pilots | Centralized | Scarce competence is pooled instead of scattered; standards are created once instead of three times |
| Several units with their own production use cases, platform established | Hybrid | The center secures standards and the platform, business units deliver products with business proximity |
| High AI maturity in several units, strong decentralized teams | Federated with guardrails | Central steering would slow things down; binding guardrails keep the whole consistent |
| Heavily regulated business (finance, health) | Centralized or hybrid with central review and approval functions | Compliance, security, and model-risk reviews must remain consistent and independent |
The organizational form is not a one-time decision: most organizations start centralized and federate as maturity grows. The signal for the transition comes from the maturity profile (see chapter A3): if business units repeatedly wait for the center even though they have the competence, the structure is slowing things down rather than securing them.
The AI competence center in concrete terms
The AI competence center is the central team that bundles platform, standards, and support. For the centralized and the hybrid form it is the core of the operating model. The federated form rarely does without one either: there the center is smaller, but it holds the shared guardrails, mandatory reviews, and platform building blocks the business units build on. So if you work federated, you use this section to size that small core; if you work centralized or hybrid, for the whole team.
To make the center workable, four things must be determined: its mandate, its staffing, its funding, and its success criteria. Together they form the center’s profile.
Mandate (service catalog). The mandate defines what the center is responsible for. The tool for it is a service catalog: a binding list of the services the center offers the business units, the way a service provider describes its offering. What to determine is which of these services your own center takes on and commits to. The benefit is twofold: the business units know what they may expect and where to get it, and the center is protected from becoming a diffuse contact point for “everything AI.” Whatever is not in the catalog is not its job. The table shows the usual five service blocks as a starting point:
| Service | Content |
|---|---|
| Platform services | Model access (gateway), RAG infrastructure, development environments, monitoring (see chapter B5) |
| Delivery support | Implementing use cases with the business unit, from discovery to handover |
| Review and approval services | Coordinating security, regulatory, and ethics reviews (see chapters B8 to B10) |
| Standards and templates | Maintaining model cards, evaluation suites, the prompt library, reference architectures |
| Enablement | Training paths, communities of practice, advising business units (see chapter B11) |
Initial staffing. Small and complementary instead of large and homogeneous: a lead with business and technology understanding, 1 to 2 data/ML engineers, 1 data scientist or GenAI expert, and fractional architecture, security, and compliance competence from the existing functions. Growth follows demand, not the org chart.
Funding. Three basic patterns, often combined: a central budget for platform and standards (common goods), charging delivery services to the commissioning business units, and usage-based allocation of model and platform costs (showback/chargeback, see chapter D2). Pure central funding turns the center into a free service provider without prioritization pressure; pure chargeback strangles the early, uncertain initiatives.
Success criteria. The competence center is measured by the same standards as the program: lead time from idea to pilot, share of reused building blocks, business-unit satisfaction, time to approval. A center that only keeps itself busy shows up in these metrics.
Intake: from idea into the portfolio
The two previous sections built the organization: the structure and the team within it. From here the perspective shifts to a single initiative and the path it takes through that organization. It starts at intake, the defined path by which ideas enter the portfolio. Without it, steering happens by acclamation: whoever asks loudest gets served.
| Step | What happens | Who |
|---|---|---|
| 1. Submission | The idea is submitted with the use-case profile (see chapter B2); deliberately low-threshold | Business unit |
| 2. Screening | Short check: ethics screening (see chapter B10), rough regulatory classification (see chapter B8), data availability | AI competence center, 1 to 2 weeks |
| 3. Assessment | Value × feasibility scorecard together with the business unit | Competence center + business unit |
| 4. Portfolio decision | Acceptance, rejection, or waiting list; slotting into the roadmap wave (see chapter C2) | Portfolio Board (see chapter B7) |
Review depth scales with risk: the intake does not treat all ideas the same. Low-risk use cases pass screening and approvals in an abbreviated procedure: the ethics screening ends without conditions for them, and minimal regulatory risk triggers no special obligations. The full review scope is reserved for the sensitive cases. That way governance protects without slowing down low-risk initiatives.
Rejections are documented and communicated too: a transparent no keeps submitters in the process, a silent fizzle drives them into shadow AI (see chapter A1).
Lifecycle management of AI models
Every AI model travels the same path: it starts as an idea, is developed and trained, tested, goes into operation, and is eventually decommissioned. The operating model determines that none of these transitions just happens; each one requires a binding approval by a named function. That is exactly what separates managed operations from sprawl: without approvals, an unreviewed model ends up in production, and a switched-off one leaves undeleted data behind. The figure shows the path with its approval points; the table below names the approval and the responsible function per transition. For generative models and agents, LLMOps and AgentOps practices apply in addition (see chapter A2).
| Lifecycle step | Binding approval | By |
|---|---|---|
| Idea → development | Portfolio decision (intake above) | Portfolio Board |
| Development: data access | Data approval with purpose limitation | Data owner (see chapter B7) |
| Test → production | Security, compliance, and ethics approval; evaluation evidence against thresholds | Independent review functions (see chapters B8 to B10, D1) |
| Operation: substantial change (new model, new data source) | Renewed short review; for high-risk, check whether a new conformity assessment is needed | Service owner + review functions |
| Operation → decommissioning | Orderly teardown: shutdown plan, data deletion per retention rules, archiving of evidence | Service owner + data owner |
Ongoing monitoring (correct function, no bias, timely renewal) belongs to the service owner and is backed with metrics in chapter D1.
What is binding for every initiative: standards, governance, and partners
This ends the path of the single initiative, and the view returns to the organization. This section does not define what one particular model must do, but which rules the organization makes binding for every initiative. Three determinations belong here: which working practices are mandatory (standards and tools), who oversees compliance (governance and compliance), and how external partners are steered (partners and supply chain). They apply in every organizational form, including the federated one.
Standards and tools
Scalable operations need repeatable processes for data preparation, training, deployment, and monitoring. The operating model defines which standards are binding; everything else remains a team decision. A proven minimum set:
- Versioning of code, data, models, and prompts; reproducible builds
- Model and data card per production model
- Evaluation suite with thresholds before every deployment (see chapter D1)
- Deployment only via the CI/CD pipeline, no manual production changes
- Monitoring incl. drift and costs per use case (see chapter D2)
Deviations are possible but require justification and documentation. That keeps the standard enforceable without blocking special cases.
Generate evidence automatically: whoever enforces these standards through the CI/CD pipeline receives the compliance evidence as a by-product: model card, evaluation report, and logs are generated and archived in versioned form with every deployment, instead of being maintained by hand. That reduces the documentation effort considerably and gives auditors the current state at any time instead of an outdated paper file.
Governance and compliance
Operations require clear rules for security, data protection, and ethics, as well as implementation of regulatory requirements (EU AI Act, industry-specific standards, see chapter B8). The model defines audits, risk analyses, and documentation obligations, and names who monitors compliance. The principle of segregation of duties applies structurally: review and approval functions are organizationally separate from development and operations (see chapter B7, segregation of duties), including in the federated model.
Partners and supply chain
Many companies use external partners (cloud, consulting, specialized platforms). The operating model defines selection, steering, and monitoring, including handling dependencies, exit strategies, and quality and security requirements (selection criteria and question catalog: chapter B3; contract details: chapter D2). For regulated industries, the supervisory outsourcing and third-party requirements come on top.
Steering and evolving the operating model
The operating model is not a project with an end date. Three tasks keep it effective over time: measuring its results (success measurement), securing the necessary capabilities (competency and knowledge), and regularly sharpening processes and standards (continuous improvement).
Success measurement
Two things have to be measured here, and they are easily confused.
- The AI solutions: the operating model defines which technical KPIs (such as model quality, availability) and business KPIs (cost reduction, revenue, customer satisfaction) are collected per initiative. A transparent reporting mechanism allows management to track progress and steer accordingly (metrics system: chapter D1; reporting paths: chapter D4).
- The operating model itself: lead times per lifecycle step and approval durations show whether the processes carry or brake.
Competency and knowledge
An operating model only works with the right capabilities. Programs for training and continuing education, as well as communities of practice, spread and secure knowledge and reduce dependency on individuals. The talent strategy incl. the competency matrix is developed in chapter B11; the competence center carries enablement as a service in its catalog (above).
Continuous improvement
Technology, data landscapes, and regulation keep changing. The operating model is therefore itself reviewed regularly: an annual effectiveness review of governance, quarterly evaluation of lead times and bottlenecks, and a defined path for teams to propose improvements to standards and processes. Communities of practice carry best practices between units.
Checklist: viable operating model
The checklist verifies that the operating model answers the six questions from the start of the chapter:
- Organizational form deliberately chosen and justified; transition signals (maturity) defined
- AI competence center set up with service catalog, initial staffing, funding model, and success criteria
- Intake process established from submission to portfolio decision; review depth scaled by risk; rejections are communicated
- Binding approvals defined per lifecycle step (table above)
- Binding minimum set of standards established; deviations require justification; evidence generated automatically from the pipeline
- Governance and compliance accountability assigned; review functions independent
- Partner-selection and exit rules in place
- Reporting mechanism established; the operating model’s own lead times measured
- Competency building planned; knowledge secured through communities of practice
- Mechanism for continuous improvement anchored