Browsing Tag
vLLM
8 posts
Ollama vLLM llama.cpp: which local engine should you choose for LLMs?
Ollama vLLM llama.cpp: compare these three local inference engines by use case, hardware and monitoring (metrics, health checks, traces).
vLLM LLM inference: install and monitor the server
vLLM LLM inference, open source: install steps from the docs, architecture, service metrics (queue, KV cache, TTFT) and OpenTelemetry traces to watch.
Open source LLM comparison: licence, size, VRAM and throughput measured on vLLM and Ollama
Open source LLM comparison: licence, parameters, GPU memory and Ollama tags for 8 verified models, plus a method to measure throughput on vLLM yourself.
Choosing an LLM in October 2026: operations criteria before benchmarks
Choosing an LLM in 2026 on operations criteria: latency, hosting, GDPR, OpenTelemetry observability and cost per task, tested with a real pilot.
ONLYOFFICE local AI with Ollama or vLLM: architecture, configuration and monitoring
ONLYOFFICE local AI: connect Docs 10.0 to Ollama or vLLM, tune aiSettings and the /ai-proxy relay, then monitor TTFT and StatsD with OpenTelemetry.
Mistral Large 4 self-hosting: what is published, sizing and monitoring
Mistral Large 4 self-hosting: weights not yet released, memory to plan for 1 trillion parameters, and vLLM multi-GPU monitoring for operations teams.
Self-hosted open source AI stack: comparing the building blocks of a self-hosted AI infrastructure
Compare the building blocks of a self-hosted open source AI stack: Ollama, vLLM, Open WebUI, LlamaIndex, CrewAI, Dify, n8n, Langfuse and typical setups.
vLLM production metrics: running an OpenAI-compatible server with Prometheus and GPU monitoring
vLLM production metrics: deploy an OpenAI-compatible server and track queue, KV cache, TTFT and GPUs with Prometheus, OpenTelemetry and Dynatrace.