A2. Foundations and Terminology

Anyone discussing architecture, operations, cost, or regulation needs clear terminology, otherwise business, IT, and the board talk past each other.

What this chapter delivers: the shared language for everything that follows. The opening parts frame the key concepts, and the closing glossary defines the terms used throughout the work. The chapter is deliberately designed as a reference section.

AI, machine learning, deep learning, generative AI

These four terms are often used interchangeably in everyday speech but describe nested concepts:

• Artificial Intelligence (AI): the umbrella term for systems that take on tasks normally requiring human intelligence. This includes classic rule-based systems.
• Machine Learning (ML): the subfield in which systems learn patterns from data instead of being explicitly programmed. Most productive “AI” applications are ML.
• Deep Learning: an ML approach based on deep neural networks. It dominates wherever unstructured data (image, speech, text) is processed.
• Generative AI (GenAI): creates new content such as text, images, code, or audio. It is based on large, pre-trained foundation models.

For strategy, the distinction between predictive and generative is crucial: predictive ML models (e.g., fraud detection, sales forecasting) return a number or class and are straightforward to measure; their typical risks are biased training data, drift, and silently declining accuracy (see chapters D1 and B10). Generative models produce open-ended output and are harder to evaluate. On top of the risks above, they carry two that only exist for them: hallucination can only occur when a model produces freely formulated content, and prompt injection requires a system that interprets free-form input as instructions; neither applies to predictive models. Both kinds of model belong in an AI strategy but require different architecture, governance, and metrics.

Language models (LLM) and foundation models

A Large Language Model (LLM) is a generative foundation model for language. It is pre-trained on very large amounts of text and can then be used for many tasks without being retrained for each one. Two sourcing models shape every architecture decision today:

• Proprietary models via API (e.g., from major providers): quickly available, high quality, but ongoing per-request costs, vendor dependency, and data-flow questions.
• Open (open-weight) models for self-hosting: full data sovereignty and predictable costs, but own operational effort and hardware requirements.

This choice is not a purely technical question. It touches data protection, cost model (FinOps), vendor lock-in, and sovereignty. Chapter B3 (Build, Buy, or Partner) provides the decision criteria, chapter B5 (Target Architecture) anchors it technically in the deployment model (cloud, on-premises, hybrid), chapter D2 (Cost and FinOps) covers the cost implications, and chapter B8 (Regulation and Compliance) covers the data protection questions.

The three ways to give a model knowledge and behavior

One of the most common and expensive mistakes is training a model where a simpler approach would have sufficed. There are three basic approaches, which can be combined:

Rule of thumb: start with prompting, then RAG, then fine-tuning. Only move up a level once the previous one has demonstrably fallen short. RAG is the right starting point for most enterprise knowledge applications because it combines currency and traceability without retraining models.

Agentic AI: AI agents

An AI agent is a system that doesn’t just respond but autonomously pursues multi-step tasks: it plans, calls tools (search, databases, business systems, other services), evaluates intermediate results, and keeps acting until a goal is reached. The difference from a classic chatbot is the shift from “giving an answer” to “getting a task done.”

Core building blocks of an agent:

• Planning: breaking a task down into steps.
• Tool use: controlled access to APIs, data sources, and actions.
• Memory: context maintained across multiple steps.
• Orchestration: interplay of multiple agents or tools.

Agents increase value but also risk: whoever can act can cause harm. Three guardrails are non-negotiable and are covered in depth in chapters B9 (security) and B7 (roles):

• Human-in-the-loop for any consequential or irreversible action (approvals, payments, shipping, deletions).
• Tightly scoped rights: an agent gets only the tools and access it strictly needs.
• Complete logging of every action for traceability and audit.

Operating disciplines: MLOps, LLMOps, AgentOps

To keep models from getting stuck in pilot status, defined operating practices are needed, the counterpart to DevOps for software:

• MLOps: versioning, training, deployment, monitoring, and retraining of classical ML models.
• LLMOps: additions for generative models: prompt and context management, evaluation of open-ended output, hallucination and cost control, RAG pipelines.
• AgentOps: monitoring and steering of agents: tool calls, action chains, intervention points, error handling.

These terms are given concrete building blocks and metrics in chapters B5 (Target Architecture), B6 (Operating Model and Organization), and D1 (Metrics and Evaluation).

Glossary

Alphabetical short definitions of terms used throughout the work.

• A/B test: comparison of two variants in live operation, to objectively measure the added value of an AI-based solution against a control group.
• Agent / Agentic AI: a system that autonomously plans and executes multi-step tasks via tools.
• Bias: systematic skew in data or model that leads to unfair or incorrect results for certain groups.
• Drift: divergence between current input data and training data (data drift) or degradation of model quality over time (model drift). A trigger for retraining.
• Embedding: a numerical representation of text, image, or other data that makes semantic similarity measurable; the basis for search in RAG systems.
• Evaluation: systematic assessment of the quality, benefit, and risk of an AI application (see chapter D1).
• Feature: a property derived from raw data that a model can use (e.g., “age” from date of birth).
• Feature store: a central, versioned repository of reusable features with owners and quality tests.
• Fine-tuning: retraining a pre-trained model with proprietary data.
• Foundation model: a large, broadly pre-trained model that serves as the basis for many applications.
• GPAI (General-Purpose AI): an AI model with a general purpose; a distinct regulatory category under the EU AI Act.
• Grounding: anchoring generative output in verifiable sources to reduce hallucination.
• Hallucination: factually incorrect but plausibly worded output from a generative model.
• Human-in-the-loop: mandatory human review or approval before a consequential action.
• Inference: using a fully trained model to generate output; the source of ongoing operating cost.
• Context window: the maximum amount of text a model can consider per request; limits how much knowledge can be supplied.
• LLM (Large Language Model): a large generative language model.
• LLMOps / MLOps / AgentOps: operating practices for generative models, classical ML models, and agents respectively.
• Model card: documentation of a model: purpose, data, assumptions, limits, risks. A mandatory governance artifact.
• Data card: analogous documentation of a dataset.
• Prompt: input to a generative model describing the task and context.
• Prompt injection: an attack in which malicious instructions embedded in input or referenced content hijack the behavior of a model or agent.
• RAG (Retrieval-Augmented Generation): a method that searches relevant internal documents at runtime and supplies them to the model as context.
• Token: the smallest processing unit of a language model (word fragment); the basis for billing and cost with API-based models.
• Vendor lock-in: dependency on a provider that makes switching expensive or practically impossible.

Note: regulatory terms (provider, deployer, high-risk, Annex III, conformity assessment) are defined in chapter B8 (Regulation and Compliance) and explained there in context.