Mistral Large 4 self-hosting is the question platform teams have been asking since the announcement on 6 October 2026. Before ordering GPUs, separate what has been published from what has been promised. At the time of writing (15 October 2026), Mistral offers a preview of the model through its API and the Mistral Studio platform, and announces the weights “by the end of the month”. No weights repository exists yet under the mistralai Hugging Face account, and the licence has not been published.
So you cannot self-host Mistral Large 4 yet. You can prepare the ground, though: estimate the memory needed from the official figures, lean on what Mistral documented for Mistral Large 3, and set up monitoring for a multi-GPU inference server. That is what this operations-focused guide covers.
This topic does not depend on Dynatrace: the metrics mentioned are in Prometheus format and can be collected by any backend, including Dynatrace SaaS and Dynatrace Managed.
Mistral Large 4 self-hosting: what is published as of 15 October 2026
Mistral’s official announcement gives the following:
- a model of about 1 trillion parameters, with 52 billion active per token: a mixture-of-experts (MoE) architecture;
- a natively multimodal model (text and image input) that combines instruction following and reasoning in a single model;
- training on 3,800 NVIDIA Grace Blackwell GPUs in Mistral’s own data centres;
- an API preview, weights promised for the end of the month, and more architecture details at that point.
What is still missing for serious sizing: the licence, the context length, the published weight formats (FP8, NVFP4 or other) and the supported vLLM version. Until these are published, any hardware figure is an estimate.
Estimating memory for a 1-trillion-parameter model
In an MoE model, all experts must sit in GPU memory, even though only a few work on each token. Memory therefore depends on the total parameter count; compute per token depends on the active parameters. The weights alone are easy to estimate:
weights (bytes) = total parameters × bytes per parameter
1 trillion × 2 bytes (BF16) ≈ 2 TB
1 trillion × 1 byte (FP8) ≈ 1 TB
1 trillion × 0.5 byte (4-bit) ≈ 0.5 TB
Add the key-value cache (KV cache), which grows with context length and the number of concurrent requests, plus the inference engine’s own overhead. These orders of magnitude are enough for a first conclusion: even quantised, Mistral Large 4 targets one or more data-centre GPU nodes, not a workstation. For local use, smaller models served by Ollama remain the realistic route.
The published reference point: Mistral Large 3
The Hugging Face model card for Mistral Large 3 (675 billion parameters in total, 41 billion active, Apache 2.0 licence, 256k-token context) gives official deployment figures:
- the FP8 version runs on a single node of B200s or H200s, with an example on one 8xH200 node;
- the NVFP4 version runs on a single node of H100s or A100s.

At equal precision, Large 4 has about 1.5 times as many parameters as Large 3. Wait for the official model card before concluding that one node will do: the published quantisation format changes everything. The Large 3 card also notes that --max-model-len can be lowered to save memory: that is the first lever when the KV cache runs short.
In practice: preparing for self-hosting
Step 1 — Test on the API preview
Measure your real use cases on the API now: average prompt length, response length, need for long context, tool calls. These measurements set the context length to configure and the concurrency to plan for, and therefore the size of the KV cache.
Step 2 — Validate the pipeline with Mistral Large 3
The deployment pipeline (weight storage, download, start-up, health probes, monitoring) can be rehearsed with Large 3, whose weights and licence are published. Across several GPUs, vLLM splits the model with tensor parallelism (--tensor-parallel-size) and, when needed, pipeline parallelism (--pipeline-parallel-size). On a single node it uses multiprocessing; across nodes, Ray.

This diagram shows what to monitor: the API server (HTTP input, tokenisation), the engine core (scheduler and KV cache) and one process per GPU. A failed GPU stops the whole instance: with tensor parallelism there is no partial degradation.
Step 3 — Wire up monitoring
vLLM exposes Prometheus metrics on /metrics, with the vllm: prefix. For a large MoE model, watch these first:
vllm:kv_cache_usage_perc: fraction of the KV cache in use (0 to 1). Close to 1, requests start waiting;vllm:num_requests_running: requests in progress;vllm:time_to_first_token_seconds: time to first token (TTFT), sensitive to prompt length;vllm:inter_token_latency_seconds: delay between tokens, felt directly when streaming;vllm:request_queue_time_secondsandvllm:e2e_request_latency_seconds: waiting time and total duration;vllm:prompt_tokens_totalandvllm:generation_tokens_total: volume processed, for cost-per-token calculations.
Collection and dashboards are covered in our vLLM in production guide. Add GPU metrics (memory, temperature, utilisation): on an eight-GPU node, a single overheating component slows down the whole batch.
How to check that it works
curl http://localhost:8000/v1/modelsreturns the name of the served model.curl -s http://localhost:8000/metrics | grep vllm:kv_cache_usage_percreturns a value between 0 and 1.- Under test load, p95 TTFT stays below your target and
vllm:kv_cache_usage_percdoes not stay pinned at 1.
Best practices
- Do not lock in any hardware budget before Mistral publishes the Large 4 model card and licence.
- Define SLOs before deploying: TTFT, inter-token latency, availability. They set the concurrency you can accept.
- Plan for start-up time: loading a model this size takes a while, which lengthens recovery after an incident (MTTR).
- Store the weights close to the nodes: downloading 1 TB again at every restart is not a strategy.
- Compare with the API: depending on your volume, a dedicated node may not pay off. Our overview of self-hosted AI stacks helps you decide.
Frequently asked questions
Can I download Mistral Large 4 today?
No, not as of 15 October 2026. Mistral announces the weights for the end of the month; only the API preview is available.
Will Mistral Large 4 be licensed under Apache 2.0?
That has not been published. Mistral Large 3 is, but nothing guarantees that Large 4 will follow. Read the model card when it ships.
Why do 52 billion active parameters not reduce memory?
Because the router can pick any expert for each token: all of them must be loaded. Active parameters reduce compute, not weight memory.
Conclusion
Mistral Large 4 cannot be self-hosted yet, but its scale is known: about 1 trillion parameters, so at least one data-centre GPU node. Use the wait to measure your usage on the API, rehearse the pipeline with Mistral Large 3 and set up monitoring. To go further on what this kind of workload demands, read what AI workloads demand from SREs. Sources: Mistral Large 4 announcement and Mistral Large 3 model card.
span.observer is an independent community blog, not affiliated with Dynatrace LLC or Mistral AI. Trademarks belong to their respective owners. Tools change quickly: check commands and options in their official documentation.