Cloud Server for LLM Hosting in Europe: GDPR AI Guide
Self-hosting a large language model gives you complete control over what data enters the model, where it is processed, and who can access it. For European businesses, this is not just a cost argument - it is a compliance requirement. Any prompt that contains personal data about EU residents must be processed under EU jurisdiction under GDPR.
This guide covers the hardware needed to host LLMs in production, how to choose between model sizes and quantization levels, and which serving frameworks work best on EU cloud infrastructure.
Why EU Jurisdiction Matters for LLM Hosting
When users interact with an LLM - asking questions, getting documents summarized, generating content - those prompts frequently contain names, email addresses, health queries, and other personal data. Sending these prompts to a US-hosted API means personal data leaves EU jurisdiction on every request, creating ongoing compliance exposure.
Self-hosting on a DCXV EU cloud server means all inference stays within EU borders. No transatlantic data transfer, no reliance on standard contractual clauses, and no dependency on a third-party provider's data processing practices. For healthcare, legal, and financial applications in Europe, self-hosted EU LLM infrastructure is the practical path to GDPR compliance.
Network latency is also a factor. A self-hosted LLM in Prague or Frankfurt adds 5-15ms to your application's inference path. The same model accessed via a US API endpoint adds 80-120ms per call - enough to degrade the experience in chat interfaces and real-time copilots.
Choosing Model Size and Quantization
The right model depends on your use case and available hardware:
- 7B models (Q4 quantized, ~4 GB VRAM) - suitable for summarization, classification, Q&A on documents. Runs on a single consumer GPU or a high-core-count CPU server.
- 13B models (Q4 quantized, ~8 GB VRAM) - stronger reasoning, better instruction following. Requires a mid-range GPU (RTX 3090/4090) or 2x smaller GPUs.
- 34B models (Q4 quantized, ~20 GB VRAM) - near-GPT-3.5 quality. Requires a single high-VRAM GPU (A100 40 GB) or two 24 GB GPUs.
- 70B models (Q4 quantized, ~40 GB VRAM) - GPT-4 class for many tasks. Requires an A100 80 GB or two A100 40 GB in tensor parallelism.
Quantization (INT4/INT8 via GGUF or GPTQ) reduces VRAM requirements by 2-4x with modest quality loss - acceptable for most production use cases.
Minimum Specs for LLM Hosting
- CPU serving (7B Q4 model) - 16 vCPU, 32 GB RAM, 200 GB NVMe SSD
- GPU entry (7B-13B, single RTX 4090) - 8 vCPU, 32 GB RAM, 24 GB VRAM, 500 GB NVMe
- GPU mid (34B Q4, single A100 40 GB) - 16 vCPU, 64 GB RAM, 40 GB VRAM, 1 TB NVMe
- GPU high (70B Q4, A100 80 GB) - 16 vCPU, 128 GB RAM, 80 GB VRAM, 2 TB NVMe
Recommended DCXV Configuration
DCXV cloud servers provide GPU-equipped EU servers for LLM hosting with Tier III certified infrastructure and private networking:
- GPU server, 24 GB VRAM - 7B-13B models at FP16 or 34B models at INT4, for SaaS copilots and internal assistants
- GPU server, 80 GB VRAM - 70B models at INT4 or 34B at FP16, for high-quality production APIs
- CPU server, 32-64 GB RAM - 7B models at INT4 via llama.cpp, for background processing and batch jobs
Contact sales@dcxv.com for GPU availability and to discuss multi-GPU tensor parallelism for larger models.
Quick Setup Commands
# Option 1: Serve with Ollama (simplest, CPU and GPU)
curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama
# Pull a model and expose as API
ollama pull llama3.1:8b
# Expose on private network:
# Add to /etc/systemd/system/ollama.service:
# Environment="OLLAMA_HOST=0.0.0.0:11434"
sudo systemctl daemon-reload && sudo systemctl restart ollama
# Test
curl http://10.0.0.5:11434/api/chat -d '{
"model": "llama3.1:8b",
"messages": [{"role": "user", "content": "Summarize GDPR in 3 bullet points"}]
}'
# Option 2: vLLM for high-throughput GPU serving (OpenAI-compatible API)
pip install vllm
python -m vllm.entrypoints.openai.api_server \
--model meta-llama/Llama-3.1-8B-Instruct \
--host 10.0.0.5 \
--port 8000 \
--tensor-parallel-size 1 \
--max-model-len 8192 \
--gpu-memory-utilization 0.90 \
--served-model-name llama3-8b
# Test with OpenAI-compatible client
curl http://10.0.0.5:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "llama3-8b", "messages": [{"role": "user", "content": "Hello"}]}'
# Option 3: llama.cpp server for CPU or GPU (lowest memory overhead)
sudo apt install -y build-essential cmake libcurl4-openssl-dev
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
# Build with GPU support (CUDA)
cmake -B build -DGGML_CUDA=ON
# Or CPU-only with AVX-512:
cmake -B build -DLLAMA_AVX512=ON
cmake --build build --config Release -j $(nproc)
# Serve a GGUF model
./build/bin/llama-server \
--model /models/llama-3.1-8b-instruct-q4_k_m.gguf \
--host 10.0.0.5 \
--port 8080 \
--ctx-size 8192 \
--n-gpu-layers 35 \
--parallel 4
Expected Performance Benchmarks
vLLM on RTX 4090 (24 GB VRAM), Llama 3.1 8B FP16:
- Single-request generation - 80-120 tokens/s
- Batched throughput (8 concurrent requests) - 400-700 tokens/s total
- Time to first token - 150-300ms
vLLM on A100 80 GB, Llama 3.1 70B INT4:
- Single-request generation - 25-40 tokens/s
- Batched throughput (4 concurrent) - 100-180 tokens/s total
- Time to first token - 300-600ms
llama.cpp CPU (16 vCPU), 8B Q4_K_M:
- Generation speed - 18-30 tokens/s
- Time to first token - 800ms-2s
Bottom Line
Self-hosting LLMs on EU cloud infrastructure is the most reliable path to GDPR-compliant AI in production. Choose your model size based on quality requirements and budget, use quantization to reduce VRAM needs, and pick vLLM for GPU production serving or llama.cpp for CPU flexibility. DCXV provides the GPU servers and EU data residency you need to run LLMs compliantly at scale.
