On-premise AI in Frankfurt: open-weight models in your own server room or rack
I plan and run on-premise AI for companies in Frankfurt Rhine-Main: open-weight models such as DeepSeek, Qwen, Mistral or gpt-oss in your server room or in a colocation rack in the region. I size the hardware for the rack and set up a fallback and access rules. After the handover your team runs the model, or I stay on for the operation. I take on freelance and interim engagements. Book a 30-minute video call. We look at your task, and you find out whether I can help.
What I take care of for you
-
Experience from the rack up
I have installed servers, switches and firewalls in data centers in Germany and the United States and cabled the racks myself. I plan your model's hardware for the staff who will use it.
-
Power checked before the purchase
One GPU card can draw up to 600 watts, an eight-GPU system about 14.3 kilowatts. I compare these figures with what your room or rack supplies before anything is ordered.
-
A second site
I moved a node operation from Azure to another cloud, with a second site that takes over when the first fails. Your model gets a fallback for the hours a server is down.
-
Access under rules
Each application reaches the model with its own token, and every request lands in a log. Agents get the rights of a new colleague on the first day, as in my client work.
-
A model you can swap
My agents run on Claude and Codex under one rule file, with rules and memory outside the model. An open-weight model can take a cloud model's place in the same setup.
-
Operation that stays with you
Your team receives a runbook for updates, restarts and model changes. If nobody in house can run the model yet, I stay on as a freelancer until someone can.
What on-premise AI means for a company
On-premise AI is a language model that runs on servers your company owns, in your own server room or in a rented rack at a colocation provider. The weights, the prompts and the answers stay on those machines. DeepSeek, Qwen, Mistral and OpenAI’s gpt-oss publish their trained weights, and open-source agent software can use such a model where it would otherwise call a cloud model.
Choosing the model is one decision. The others concern the room around it: power, cooling, network, a fallback, access rules and a person who keeps the model running. This page is about the rack and the operation, for companies in Frankfurt, Offenbach, Wiesbaden, Darmstadt, Mainz, Hanau and Bad Homburg.
From model to rack, checked in September 2026
The size of the model decides how much GPU memory you need. The GPUs decide how much power and cooling the rack has to supply. The table sets published figures side by side.
| Model or hardware | Published figure | What it means for your rack |
|---|---|---|
| gpt-oss-20b (OpenAI) | 21 billion parameters, runs within 16 GB of memory | Fits on one GPU in an ordinary server. |
| gpt-oss-120b (OpenAI) | 117 billion parameters, fits on a single 80 GB GPU | One data center GPU in one server. |
| DeepSeek-V4.1-Flash | 552 billion parameters | At one byte per parameter the weights alone take about 552 GB, more than any single GPU in this table holds. |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB GPU memory, up to 600 W, air or liquid cooled | Four cards draw up to 2.4 kW before the rest of the server. |
| NVIDIA DGX B200 | 8 GPUs, 1,440 GB GPU memory, about 14.3 kW, 10 rack units | Ask the provider whether one rack can supply 14.3 kW and remove the heat. |
Sources: gpt-oss, DeepSeek-V4.1-Flash, RTX PRO 6000 Blackwell Server Edition, DGX B200. The number of people who ask at the same time and the length of their documents add to the memory a model needs. The assessment estimates both from your tasks before anything is bought.
What to ask your server room or colocation provider
Before a GPU server moves in, the room has to answer these questions. I go through them with you or with the provider:
- How many kilowatts does one rack get, and does that cover the GPU servers at full load?
- Can the cooling remove that heat, by air or by liquid?
- Is there a second power feed and a UPS, so a power cut does not stop the model?
- How fast is the line to your offices, and does the data center connect to an internet exchange such as DE-CIX?
- Who may enter the room, and is every entry logged?
- Can the provider’s technicians swap a part when nobody from your company can drive there?
- Which second machine, second site or approved cloud model takes the requests when the first site fails?
DE-CIX says its Frankfurt exchange connects more than 1,000 networks and can be reached in more than 30 data centers (DE-CIX Frankfurt).
What runs after the setup
A model in production needs the same care as your other servers. I set up:
- monitoring of GPU memory, temperature and response times,
- a log of every request with the application that sent it,
- a plan for model updates: the new version runs beside the old one until your staff approve it,
- a runbook your IT team can follow when something fails.
Agents that use the model follow the same rules as humans.
Who I am
I am Michael Wutzke, an IT expert for data platforms in Frankfurt. I have worked more than twenty years from code to CIO in finance, e-commerce and real estate, with cloud platforms and data centers. Today AI agents speed up how I build and transform data platforms.
In client work I rebuild a platform with Claude and Codex under one rule file. The rules and the memory of the agents stay outside the model. An open-weight model on your own servers can then do the work of a cloud model. In data centers in Germany and the United States I set up servers, switches, firewalls and backup systems and did the cabling. For a web community I set up 40 load-balanced servers myself. I have run blockchain nodes in several data centers. I moved infrastructure from Azure to another cloud with failover.
How your model gets into the rack
-
Video call
In 30 minutes we talk about the tasks for the model, the data it would see and where your servers stand today.
-
Assessment together
We go through your server room or your colocation contract: power per rack, cooling, network and physical access. The model size and the number of users decide the GPUs.
-
Quote and order
You receive a quote for the setup and, if you want it, for the operation. The work begins when you accept it.
-
Setup
I install the servers myself or brief the provider's technicians. Then I set up the serving software, a token per application and the request log.
-
Failover test
We switch off one server on purpose and check that requests reach the fallback, a second machine or a cloud model you approved.
-
Handover
Your team receives the runbook and decides who operates the model. I can stay on for the operation, part time.
Questions companies ask
Should we run AI on premise or in the cloud?
That depends on your data and your load. Data that must not leave the company speaks for your own rack. A model that works only a few hours a week can cost less as a cloud service. The assessment compares both for your tasks.
Can our server room hold a GPU server?
A single card can fit where a larger system does not. NVIDIA lists the RTX PRO 6000 Blackwell Server Edition at up to 600 watts and the DGX B200 at about 14.3 kilowatts in 10 rack units. I compare these values with the power and cooling your room supplies.
Why a colocation provider in Frankfurt?
At a colocation provider you rent rack space with power, cooling and access control, and the servers stay yours. According to DE-CIX, its Frankfurt exchange connects more than 1,000 networks and can be reached in more than 30 data centers. I help you compare offers in the region.
Who runs the model after the setup?
Your IT team, with the runbook from the handover, or I run it as a freelancer until your team takes over. Updates to the model and to the serving software follow a written plan either way.
Which models do you work with?
In client work my agents run on Claude and Codex. For on-premise AI I bring the operation around the model: servers in data centers, blockchain nodes in several data centers and a cloud move with failover. Rules and memory stay outside the model, so an open-weight model can replace a cloud model.
Which engagements do you take on?
Freelance and interim engagements, part time or full time. I live in Frankfurt and come on site across Frankfurt Rhine-Main, to your office, your server room or your colocation provider.
Details on the work behind this page
-
Running infrastructure
More than 20 years of running IT infrastructure: on servers in office server rooms, in data centers and in the cloud.
-
Node operations and a cloud migration with failover
Blockchain nodes and an oracle network in several data centers, then a move from Azure with failover.
-
Self-hosted LLMs such as DeepSeek for companies in Germany
-
AI agents under the same rules as humans
Limits at the resource, a blocking check before each tool call, and approvals that only a person gives.
Your on-premise AI expert in Frankfurt
I am Michael Wutzke, an on-premise AI expert in Frankfurt Rhine-Main. In a 30-minute video call you learn which open-weight model fits your tasks and what your server room or colocation rack has to supply for it.
Searches this page answers
- on premise ai
- best on premise ai model
- on premise ai open source
- on premise gpu server
- on premise vs cloud generative ai total cost of ownership
- ai data center frankfurt
- colocation frankfurt
- ai colocation
- sovereign ai germany
- private ai infrastructure
- air gapped ai system
- ai server rack
- ai server power requirements
- llm failover