How to set up local inference under German data rules.

When teams say “local LLM,” they often mean a GPU box and a model card. Under German data rules the harder setup is the control boundary: which objects may enter which inference environment, who may call it, whether customer data trains anything, how long prompts and logs stay, and who approves the path.

This piece is a how-to for open-weight or local inference — and German cloud inference where you need it — so residency is not mistaken for finished compliance. It sits next to agents on Spectacle: proposals wait for human approval; arithmetic stays in code.

Four layers before any object crosses

Name four layers before prompts, embeddings, or documents leave the trust zone. The open-weight addendum locks the same cut: licence · residency · retention · approval.

1. Licence — What does the open-weight licence allow for commercial use, redistribution, and fine-tuning? Who owns the weights you run? Licence choice is an ops object, not a side note on a model card.
2. Residency — Where do weights, prompts, embeddings, and tool outputs sit: own hardware, dedicated private cloud, or a German/EU-hosted path such as STACKIT? Residency is necessary and not sufficient. Say it once: putting inference in Germany does not finish legal or operational duties by itself.
3. Retention — How long do prompts, completions, embeddings, logs, backups, and support traces live? Who can delete them? Hidden copies in vector indexes and backup tunnels count.
4. Approval — Which object classes may enter which inference environment? Who releases that mapping? Firm-wide infrastructure vs matter- or mandate-scoped paths. Approval maps to objects from the operation and, for sensitive paths, to a case graph (sensitive data needs a case graph). See without approval the agent sleeps.

That is how to set up local inference under German data rules. A model download alone is not the setup.

Checklist: access, no-train, deletion, approval

Before production traffic:

• Access control — Who may invoke which endpoint on which object class? Read vs propose vs act. Secrets and network paths named, not assumed.
• No-train — Confirm that customer and matter data are not used to train or improve the provider’s or your shared base model unless counsel and ops explicitly approve a separate training path. Default: inference only.
• Deletion / retention — Document TTL for prompts, embeddings, logs, backups. Test that delete means delete across indexes and support tunnels.
• Approval of object classes — A named person releases which objects may enter local, private-cloud, or German-cloud inference. Agents propose; humans release.

Also inventory hidden egress: prompts and system messages, embeddings and vector stores, application logs, backups and snapshots, vendor support tunnels and remote update channels. Telemetry that leaves the trust zone is still an inference-environment decision.

Where German cloud fits (without republishing the cousin)

Sovereign or German hosting such as STACKIT can be a residency option when own hardware is not the right boundary. Hosting is not the operation — models on STACKIT are not the operation. Cite only; do not republish that cousin. This piece is the local / open-weight how-to under German data rules — access, no-train, deletion, approval — not a remake of the STACKIT thesis.

Named sovereign-AI SaaS pitches and GPU shopping guides show up as SERP clutter. Note the type. Do not endorse a brand as the answer. Never invent “GDPR-compliant” or “§203-safe” claims for any stack. Residency is not compliance.

Approval still holds the switch

Agents on Spectacle may propose language and next steps grounded in objects the stack already holds. Without approval the agent sleeps. Sealed systems of record are not written back. Arithmetic stays in code. The model only writes language. We do not invent hours, ROI, euros, or accuracy bands.

What we are not selling

Not a named sovereign-AI SaaS. Not a deep-tech GPU shopping guide. Not a remake of the live STACKIT or case-graph cousins. Not “AI for hospitality” or restaurant software. Not silent write-back. Not a claim that any cloud or on-prem stack is “production-safe” or “GDPR-complete” by brand.

One core, every kind of business — one core, every operating type. Six sector bands equal. White-collar and ops paths use the same pattern — objects, zones, approval — not a sector SKU.

FAQ

What does “local inference” mean here?
Running open-weight or self-hosted models (and German cloud where needed) with a named control boundary: access, no-train, retention/deletion, and approval of which objects may enter.

Is residency enough under German data rules?
No. Say it once: residency is necessary and not sufficient. You still need access control, retention, approval, and counsel on the legal bases that apply to your path. We do not invent compliance labels for stacks.

Can we name STACKIT?
Yes as infrastructure vocabulary for a German cloud option. No as a claim that hosting alone finishes the setup. Cite the live cousin; do not republish it.

Do agents send objects into inference without approval?
No. They propose. You release which object classes may enter which environment. Without approval the agent sleeps.

Do you invent accuracy or ROI figures?
No. Arithmetic is code. We do not invent hours, euros, or accuracy bands.

Is this only for one sector?
No. Six bands equal. The four-layer setup — licence, residency, retention, approval — repeats wherever inference sits near operational objects.

Where should I start?
Spectacle. The Insights series is how we write about the operation.

CTA

If inference is about to sit near operational objects and the four layers are still unnamed — licence, residency, retention, approval — request an initial analysis. Twenty minutes is enough to see whether object classes wait for human approval before they enter local or German-cloud inference. If the data cannot answer, we say so.