B10. Ethics and Responsible AI
Regulation defines what is legally required (see chapter B8); ethics and responsible AI define what the company additionally considers right, even where the law is silent.
What this chapter delivers: the guardrails for fairness, transparency, and human oversight that already appear as principles in other chapters, bundled and made verifiable: with metrics, methods, a review process, and templates. The reference frameworks are the EU Ethics Guidelines for Trustworthy AI, ISO/IEC 42001 (AI management system), and the NIST AI Risk Management Framework; see chapter E2 for source references.
Structure of the chapter: first the five core principles, they are the yardstick. Then the ethics review with the screening that decides the review need. Use cases with at least one hit in the screening then go through stage 2, the in-depth review with the building blocks fairness, transparency and explainability, human oversight, and trade-offs. GenAI-specific questions and anchoring in the company round off the chapter.
Core principles
The five core principles apply to every AI deployment in the company, regardless of its regulatory classification. They are the yardstick that all reviews in this chapter measure against: every fairness assessment, every transparency measure, and every oversight level traces back to one of these principles. All five always apply; the third column shows in which situations a principle triggers concrete work and where that work is described.
| Principle | What the principle demands | When it triggers concrete work |
|---|---|---|
| Fairness | No system systematically disadvantages certain groups; biases are actively sought, not just handled reactively | For every use case that assesses or affects people: review before approval and continuously in operation (fairness section) |
| Transparency | Affected persons know that AI is involved and receive understandable explanations of decisions | For direct interaction with people and for decisions with external effect (transparency and explainability section; legal minimum: Art. 50, chapter B8) |
| Human oversight | Consequential decisions remain reviewable and correctable by humans | Mandatory for consequential or hard-to-reverse decisions (human oversight section) |
| Accountability | For every system, a person or function is named who answers for its impact and errors | From day one of every system, regardless of risk (roles in chapter B7) |
| Purpose limitation | AI is used for the defined purpose, not silently extended | Re-check on every change of purpose: renewed screening in the intake (see chapter B6), because a change of purpose can trigger a new regulatory classification (see chapter B8) |
The ethics review as a process
Principles only work if they are checked at defined points. The ethics review is this checking process. It is deliberately two-staged so the effort matches the risk: a short screening applies to every use case, the in-depth review only to the conspicuous ones. The screening at intake (see chapter B6) is the switch that decides which path a use case takes.
Stage 1: screening (every use case, about 15 minutes, at intake). Four questions:
- Does the system make or influence decisions about people (selection, assessment, pricing, access)?
- Does the system interact directly with customers or employees?
- Does the system process sensitive attributes or obvious proxies for them?
- Can a wrong output have consequences for affected persons that are hard to reverse (financial, health-related, professional, reputational)?
The outcome of the screening determines the path; every “yes” counts as a hit:
- No hit → stage 2 is skipped. The use case receives no ethics conditions. Continue reading: skip the stage 2 section and go straight to the section GenAI-specific ethics questions. The use case itself then goes through step 3 of the intake, the evaluation with the scorecard from chapter B2. The five core principles still apply; they simply do not trigger a separate review.
- At least one hit → continue with stage 2. The use case counts as ethically sensitive and goes through the in-depth review in the following stage 2 section before it moves into implementation.
Interlocking with the delivery model (see chapter C1): the screening belongs in the intake (Assess/Strategize phase), the in-depth review before the Pilot → Scale gate, the recurring review in operations (Operate phase). Catching up on ethics reviews after go-live is the most expensive variant: by then the model is built, the data is chosen, and every correction is a rebuild.
Stage 2: the in-depth review
Only use cases with at least one hit in the screening go through this stage; for all others it is skipped, and their path is described above at the screening outcome. The in-depth review consists of four building blocks, one subsection each: the fairness assessment, the explainability concept (transparency and explainability), setting the oversight level (human oversight), and documenting trade-offs. The result is a release recommendation by the Responsible AI officer or ethics board.
Fairness: detecting and limiting bias
Fairness is the first of the three principles that generate the most work in practice and therefore get their own section from here on. Bias usually arises through the data (historical discrimination, underrepresented groups) or through the target variable: a statistically optimal target is not automatically a fair target.
- Before training: check the representativeness of the training data (see chapter B4), identify sensitive attributes and their proxies.
- Before release: break down results by relevant groups and check for systematic deviations (metrics below).
- In operation: regular fairness reviews, not just once before go-live; re-check whenever data or model changes.
A widespread misconception first: omitting sensitive attributes (“we do not even use gender”) does not create fairness. Models reconstruct the information through proxy attributes such as postal code, occupation, or part-time ratio. Fairness has to be measured; it does not follow from ignorance.
Fairness is measurable but not unambiguous: the metric decision
There are several mathematical definitions of fairness, and they contradict each other: it is provably impossible to satisfy all of them at the same time as soon as the groups differ in their base rates. A company therefore has to decide per use case which definition is the right one, and justify and document that decision. This very trade-off is the core of fairness work; whoever skips it measures something arbitrary.
| Metric | What it demands | When it fits | Example |
|---|---|---|---|
| Demographic parity | All groups receive the positive outcome at the same rate | When access itself is the goal and historical inequality should not be perpetuated | Interview invitation rate equal across genders |
| Equal opportunity | Among those actually qualified, the detection rate is the same in all groups | When the main concern is not overlooking qualified people | Of the creditworthy applicants, the same share is approved in every age group |
| Equalized odds | Both false-positive and false-negative rates are equal per group | When both types of error cause harm | Fraud detection: no group is more often falsely suspected or falsely cleared |
| Calibration | A score means the same probability in every group | When people decide based on the score and must interpret it consistently across groups | A default score of 0.3 means a 30 % default probability in every group |
Practical selection guide: first clarify which error harms whom. If overlooking qualified people is the main harm, that leads to equal opportunity. If false suspicions cause harm, equalized odds is needed. If humans make further decisions based on the score, calibration is a prerequisite. Demographic parity is the strongest requirement and the most contested, because it can deviate from the actual distribution of qualification; it needs a deliberate business or legal justification. Additionally, set a threshold above which a deviation counts as a finding (a common rule of thumb: relative deviation above 20 %, originally from the US four-fifths rule; not prescribed as a norm in the EU, stricter depending on impact), and anchor it in the evaluation concept (see chapter D1).
Countermeasures: know the spectrum
If a bias is found, there are more options than “discard the model”. The measures differ greatly in effort and depth of intervention; often the downstream correction is the pragmatic first step and the data improvement the sustainable solution.
| Intervention point | Measure | Characteristics |
|---|---|---|
| Data (before training) | Deliberately supplement underrepresented groups, correct historically biased target variables, adjust weighting | Addresses the cause; costly, requires retraining |
| Training (in the model) | Add a fairness constraint to the optimization | Effective but methodologically demanding; not available for purchased models |
| Downstream (after the model) | Adjust decision thresholds per group, score correction, additional human review for disadvantaged groups | Quick to implement, also for third-party models; treats symptoms, clarify the legal permissibility of group-specific thresholds first |
| Organizational | Restrict the use case, raise the oversight level, exclude use for certain constellations | Always available; the right choice when technical correction is insufficient |
The data dilemma of fairness measurement
To measure bias by group, you need group membership, which is exactly the kind of sensitive attribute whose processing the GDPR (Art. 9) specifically restricts. Since the Digital Omnibus to the EU AI Act, there is an explicit legal basis for this: providers and deployers may process special categories of personal data to the extent strictly necessary to detect and correct bias. The Omnibus extends this permission, previously limited to providers of high-risk systems, to all AI systems, but ties it to strict conditions: only where the bias cannot be checked with less intrusive data (such as synthetic or anonymized data), and with strict access control, purpose limitation, a ban on onward disclosure, and deletion once the purpose is met. In data-protection terms the opening is contested; its use must be documented and aligned with data protection (see chapter B8). This dilemma is therefore solvable, but only deliberately:
- Treat fairness measurement as a purpose of its own with its own legal basis and align it with the data protection officer (document the result)
- Keep sensitive attributes only in the evaluation dataset, separate from the production system; the model itself never receives them
- Where direct collection is not possible: aggregated checks, proxy analyses, or samples with consent; document the reduced validity
- Do not use the dilemma as an excuse: “we are not allowed to measure that” otherwise ends up as “we do not know whether we discriminate”
Template: fairness assessment per use case
For every question, the table states why it must be answered. Unanswered questions here mean: the risk is unknown, not absent.
| Question | Why it must be answered | Example |
|---|---|---|
| Which groups could be particularly affected by a wrong decision? | Directs the review toward real harm instead of abstract statistics. Anyone who cannot name affected groups has not thought the use case through. | For credit decisions: young applicants without history, self-employed people |
| Which sensitive attributes (or proxies for them) enter directly or indirectly? | Discrimination rarely runs through the attribute itself but through proxies. Without proxy analysis it remains undetected. | Postal code as a proxy for origin, part-time ratio as a proxy for gender |
| Which fairness metric was chosen and why? | The metrics are mutually exclusive; without a justified choice, any measurement result is arbitrary. | Equal opportunity, because overlooking qualified applicants is the relevant harm |
| How was the distribution of outcomes checked by group? | Only a documented review holds up in an audit. “We did not notice anything” is not a review. | Approval rate compared per age group, relative deviation above 20 % flagged |
| Which deviations were found and how are they handled? | Forces a decision: fix, compensate, or accept with justification. Silent acceptance is the riskiest variant. | Retraining with more balanced data, additional human review for the affected group |
| Who is responsible for the ongoing review? | Without a named person, the review falls asleep after go-live while data and model keep changing. | Responsible AI officer, review every quarter |
Transparency and explainability
The second of the three deepened principles is transparency. It has two audiences: the affected person (do they know that AI is involved?) and the organization itself (can it reconstruct a decision after the fact?).
- Toward affected persons: labeling of AI interactions and AI-generated content, understandable explanation of the main decision factors.
- Toward the organization: model and data cards (see chapter A2) document purpose, data, assumptions, and limits; logging allows the subsequent reconstruction of a decision.
Explainability methods: which explanation for whom
“Explainable” is not a switch but a selection decision on two levels. First, the model: inherently interpretable methods (scorecards, rule sets, small decision trees) explain themselves; more powerful methods need post-hoc explanation techniques whose results are approximations, not truth. For high-impact decisions, the question “is an interpretable model sufficient?” belongs at the beginning, not at the end. Second, the audience: every target group needs a different explanation.
| Audience | Needs | Suitable method |
|---|---|---|
| Affected person | The main reasons and what they could change | Counterfactual explanation (“With an annual income above X, the loan would have been approved”), the three decisive factors in everyday language |
| Case handler / oversight person | Why this individual case was assessed this way, in order to agree or object | Local feature contributions per case (e.g., SHAP values), translated into business terms, directly in the work screen |
| Model validation / auditor | How the model behaves overall and where it tips | Global feature weights, partial dependencies, behavioral tests on edge cases, model card |
| Management / boards | Whether the model does what was decided, and which risks remain | Model card with purpose, limits, fairness results, and open risks (see chapter D4) |
Honesty with generative AI: outputs of large language models are, at the current state of the art, not explainable in the classic sense; no one can causally trace how a specific formulation came about. The substitute construct is justifiability: answers are grounded in verified sources and delivered with citations (grounding, see chapter B9), and uncertainty is disclosed rather than papered over. For use cases that require genuine explainability (e.g., adverse individual decisions), an LLM as the deciding system is the wrong choice; as a preparatory tool with human decision-making it can be permissible (see the decision tree in chapter B8).
Human oversight
The third deepened principle is human oversight. It is tiered, not binary: not every decision needs individual sign-off, but every consequential one does.
| Decision type | Form of oversight |
|---|---|
| Low impact, reversible | Sample checks after the fact |
| Medium impact | Approval by a trained person before taking effect |
| High impact, irreversible (credit, termination, medical recommendation, agent action with payment) | Mandatory individual approval by a professionally competent person, with a right to object for affected persons |
The same tiering applies to agents: tool privileges and human-in-the-loop points are assigned according to the impact of the possible action (see chapter B9).
Designing effective oversight, preventing rubber-stamping
The most common failure mode of human oversight is not its absence but its idling: the reviewing person confirms almost everything (automation bias), and oversight becomes a stamp. The obligation is then formally fulfilled, but in fact the machine decides. Effective oversight has four conditions that must be actively established:
| Condition | What breaks it | Design measure |
|---|---|---|
| Time and case load | 200 approvals per day leave no room for real review | Calculate the review effort per case and size the case load accordingly; smooth out peak times |
| Information | Only the score is visible, not the reasoning | Show the local explanation (section Transparency and explainability) and the relevant raw data directly in the review screen |
| Authority and backing | Objecting creates pressure to justify or extra work | Anchor objection as desired behavior; no obligation to justify deviations from the system’s suggestion |
| Competence | The person cannot assess the model’s limits | Role-specific training incl. the system’s known weaknesses (Art. 4 EU AI Act, see chapter B11) |
Measure instead of hope: the override rate (share of cases in which the oversight person deviates from the system’s suggestion) is collected as a metric. A rate near zero at high case volume is an alarm signal for rubber-stamping, not a confirmation of model quality. In addition, hidden control cases have proven useful: known edge cases that are interspersed to check whether oversight catches them.
Documenting trade-offs
Where fairness, accuracy, cost efficiency, and speed conflict, the trade-off is documented, not silently resolved in favor of cost efficiency. A documented trade-off has five components; the example shows the format.
| Field | Example |
|---|---|
| Conflict | The more accurate model worsens equal opportunity for applicants without credit history by 8 percentage points compared to the simpler model |
| Options examined | (1) simpler model, (2) more accurate model with group-specific post-adjustment, (3) more accurate model with human second review for the affected group |
| Decision with justification | Option 3: the accuracy gain is preserved, the affected group receives an additional review loop; post-adjustment (option 2) not conclusively clarified legally |
| Decision-maker and date | Portfolio Board on recommendation of the ethics board, 14 March 2026 |
| Review | Quarterly based on the fairness metrics and the override rate; reassessment on model change |
GenAI-specific ethics questions
The reviews so far in this chapter mainly ask whether a system treats people fairly. Generative AI adds risks that do not appear there: it can invent content freely, imitate real people convincingly, pass itself off as human, exploit others’ work, or consume a lot of energy.
The table names, per topic, the core question behind it and the matching guardrails as separate points. How to use it: for every use case involving generative AI, go through the applicable rows, regardless of the screening outcome above. Guardrails that recur are recorded once as a company-wide guideline instead of being renegotiated in every initiative.
| Topic | Core question | Guardrails |
|---|---|---|
| Hallucination (the model invents plausible-sounding but false statements) | May uncertain outputs reach affected persons? |
|
| Deepfakes and provenance | Is it recognizable what is real and what is generated? |
|
| Anthropomorphization | Does the system deliberately build emotional attachment or false trust? |
|
| Impact on employees | Does the system change or replace activities, and is that discussed honestly? |
|
| Authorship and intellectual property | Does the company exploit others’ work without attribution? |
|
| Environmental footprint | Is the resource consumption proportionate to the benefit? |
Anchoring in the company
So that the principles and reviews of this chapter do not remain a one-off effort, they need a fixed place in roles, processes, and metrics. Four anchoring points have proven effective:
- Responsible AI officer / ethics board: defines guidelines, reviews critical use cases before go-live (review process above), escalates in case of conflicting goals.
- Ethics principles as part of the release: no high-risk use case goes live without a fairness and transparency review, in addition to the regulatory conformity assessment.
- Metrics instead of declarations of intent: fairness metrics and the override rate are part of ongoing reporting (see chapter D4), not of a one-off review.
- Management system: anyone who wants to anchor responsible AI permanently can orient themselves on ISO/IEC 42001; for most companies, integrating the checkpoints into existing release and risk processes is the pragmatic path.
Checklist: ethics and responsible AI
The checklist bundles the chapter’s reviews, from screening at intake to anchoring in the company:
- Ethics screening anchored in the intake for every use case; in-depth review for hits
- Fairness assessment performed for every high-risk and customer-facing use case
- Fairness metric chosen with justification per use case, threshold defined
- Sensitive attributes and possible proxies identified; legal basis of fairness measurement clarified
- Countermeasure decided and documented per finding (fix, compensate, accept with justification)
- Explainability concept defined per audience; for generative AI, justifiability through sources
- Transparency obligations toward affected persons implemented (labeling, explanation)
- Model and data card available and current
- Oversight level defined per decision type; effectiveness conditions established
- Override rate measured and reported
- Trade-offs documented using the template
- GenAI guardrails (labeling, anthropomorphization, copyright, environment) checked
- Responsible function (Responsible AI officer or board) named
- Recurring review planned for data or model changes