D2. Cost and FinOps for AI
AI incurs costs in many places at once, from data preparation through training and operations to personnel. What sets it apart from classic software is inference, the productive model operation: every single request costs compute, so ongoing costs grow with usage. A FinOps approach applies proven cloud cost-management principles to AI: make spending visible, assign ownership, continuously reconcile cost against business benefit.
What this chapter delivers: ongoing cost control for the budget frame from chapter B1. First it introduces the eight cost fields in which AI costs arise. Two templates build on them: the cost model per use case and the unit-economics worksheet for the profitability of a single request. After that come the cost levers for reducing costs and the FinOps practice that keeps costs under control for good.
The eight cost fields
Every AI use case incurs costs in the same eight fields. Knowing all eight prevents hidden cost blocks from being overlooked and makes use cases properly comparable:
| Cost field | What belongs to it | What the amount depends on |
|---|---|---|
| Data acquisition and preparation | sourcing, cleaning, labeling, and legally clearing data | data volume, required quality |
| Development / training | experiments, model training, fine-tuning, and evaluation up to a deployable solution | number of experiments, GPU-hours |
| Inference / API | productive operation: compute or API fees for every single request | requests × tokens |
| Infrastructure / storage | compute capacity, data storage, network, and environments for development and operations | data volume, required availability |
| Licenses / platform | software and model licenses, platform and tool subscriptions | number of users, booked modules |
| Operations / monitoring | deployment, monitoring, troubleshooting, and regular retraining | solution complexity, committed service levels (SLA) |
| Security / compliance | protective measures, audits, and documentation duties | protection needs, number of audits |
| Personnel | internal roles and external support across all phases | required roles, effort |
One field stands out: inference is the only field that grows directly with usage, because every request costs anew. A successful use case therefore becomes more expensive, not cheaper. This is exactly why a one-off budget estimate is not enough for AI.
Template: cost model per use case
For every use case, all eight cost fields are estimated, separated into one-time and ongoing. The cost-driver column names what the amount depends on; if the scope changes, the estimate can be adjusted traceably. Important: an empty field means unexamined costs, not zero costs.
| Cost field | One-time | Ongoing per year | Cost driver |
|---|---|---|---|
| Data acquisition and preparation | Volume, quality | ||
| Development / training | Experiments, GPU-hours | ||
| Inference / API | Requests × tokens | ||
| Infrastructure / storage | Data volume, availability | ||
| Licenses / platform | Users, modules | ||
| Operations / monitoring | Complexity, SLA | ||
| Security / compliance | Protection needs, audits | ||
| Personnel | Roles, effort | ||
| Total |
Template: unit-economics worksheet
Total costs alone do not show whether a use case pays off. The worksheet therefore breaks the economics down to a single request; that makes visible whether the use case gets better or worse as usage grows. The same unit metrics later serve ongoing steering: cost per request, per training run, and per user; especially telling is cost per unit of result quality (e.g., per percentage point of accuracy).
| Metric | Value | Calculation |
|---|---|---|
| Cost per request | ongoing cost ÷ requests | |
| Benefit per request | time and cost saved or additional revenue | |
| Contribution margin per request | benefit minus cost | |
| Break-even volume | one-time cost ÷ contribution margin | |
| Scaling effect | cost trend at 2, 5, and 10 times the volume |
Worked example for the use case “customer-service reply suggestions” (continued from chapter B2): 200,000 requests per year, ongoing cost of 30,000 euros per year, one-time cost of 60,000 euros.
| Metric | Value | Calculation |
|---|---|---|
| Cost per request | 0.15 euros | 30,000 euros ÷ 200,000 requests |
| Benefit per request | 1.00 euros | around 1.5 minutes of handling time saved |
| Contribution margin per request | 0.85 euros | 1.00 euros minus 0.15 euros |
| Break-even volume | around 71,000 requests | 60,000 euros ÷ 0.85 euros; reached after a good four months at this volume |
| Scaling effect | contribution margin stays positive | as volume grows, the fixed-cost share per request falls and caching further lowers inference cost |
The case therefore pays off from around 71,000 requests and becomes more economical as usage grows. If the contribution margin were negative, the reverse logic of the following warning sign would apply.
Warning sign: if the contribution margin per request is negative, the use case scales into a loss zone. Higher usage makes the problem worse instead of solving it.
Cost levers
If the worksheet comes out too expensive, the use case does not have to die. The following levers reduce costs without cutting the benefit; the right-hand column shows which cost field a lever mainly acts on. Cost optimization is an ongoing process, not a one-off project (terms such as RAG in chapter A2).
| Lever | How it works | Acts mainly on |
|---|---|---|
| Caching | reuse already computed answers instead of generating them again | Inference |
| Batching | bundle many requests and process them together | Inference |
| Prompt and context optimization | shorter inputs and a leaner context reduce the token volume per request | Inference |
| RAG | grounded answers reduce costly wrong answers and rework | Inference |
| More compact models | quantization stores model weights at lower numerical precision, distillation lets a smaller model learn the behavior of a large one; both make models smaller and faster | Inference, infrastructure |
| More efficient training methods | experiment in a more targeted way, reuse already trained models | Development / training |
| Cheap compute windows | off-peak training at times of low utilization; spot instances are short-notice, much cheaper cloud capacity that the provider reclaims when needed, suitable for interruptible training runs | Development / training |
| Specialized hardware | processors built for AI workloads compute more cheaply per result | Development / training, inference |
| Cleaning up | shut down unused resources, archive rarely used data | Infrastructure / storage |
The FinOps practice: steering costs for the long run
A one-off estimate ages quickly because usage, prices, and models change continuously. The FinOps practice therefore anchors fixed habits in the organization:
| Practice | How it works |
|---|---|
| Cost transparency | capture all expenditure granularly by project, department, and technology; prevents hidden cloud or personnel costs from being underestimated |
| Cost allocation | assign costs to the business units and products that cause them rather than blanket-charging IT. Showback shows each unit the costs it caused, chargeback actually bills them to it |
| Budgets and approvals | define who approves budgets, which thresholds apply, and from when approval is required; quotas per unit, unified standards for cost reports |
| Automated monitoring | capture costs in real time, trigger alerts on overruns, throttle automatically if necessary; especially important with usage-based cloud and API pricing |
| Forecasting and budget planning | plan with scenarios (best, average, worst) and seasonal effects; makes investment plannable |
| Cost-benefit evaluation | per use case, examine return on investment (ROI), total cost of ownership (the full cost over the lifetime, not just the purchase price), and break-even (the point from which the benefit covers the one-off costs); the basis for prioritizing between use cases |
| Cost awareness | train people that every training run and every query costs money; establish guidelines for cost-efficient development |
The visible result of this practice is the monthly cost overview per use case with forecast and variance. It feeds the reporting from chapter D4.
Procurement, partners, and exit strategy
External dependencies are often underestimated. Clear criteria for selection and contracting (detailed vendor selection: chapter B3):
- Data locations meet legal requirements
- Customer data is not used to train third-party models
- Interfaces are openly documented
- Ownership of data and artifacts stays with the company
- Price-adjustment clauses and audit evidence are contractually secured
- Exit plan: export of data, features, and models regulated, replacement solution named, rollback organized
This keeps the risk of vendor lock-in (the situation where switching becomes practically unaffordable) manageable and secures long-term flexibility.
Checklist: FinOps maturity
The checklist follows the chapter’s path from estimating to lasting steering; open items are the next expansion steps:
- All eight cost fields estimated per use case, separated into one-time and ongoing
- Unit metrics defined (cost per request, training run, user)
- Unit economics per use case positive or deliberately accepted
- Cost levers anchored as an ongoing process: caching, batching, prompt and context optimization, more compact models, cheap compute windows
- Costs assigned to those who cause them, showback or chargeback established
- Budgets, quotas, and approval thresholds set
- Automated cost monitoring with alerts active
- Budget forecast produced with scenarios (best, average, worst) and seasonal effects
- Cost-benefit assessment calculated with ROI, total cost of ownership, and break-even
- Monthly cost overview with forecast and variance established
- Cost awareness trained: every training run and every query incurs cost
- Procurement and exit rules contractually secured