Browsing Tag
LLM
36 posts
Local MoE model: what Colibri asks of your disk
Local MoE model with Colibri: RAM, NVMe disk, published speeds and monitoring via /metrics. What the README says before you download 400 GB.
traceroot AI agent observability: licence, setup and limits
traceroot AI agent observability: traces, detectors and cost, a four-command Docker start, Apache 2.0 licence plus Enterprise folders, limits checked.
OpenInference Dynatrace: send AI traces through the Collector
OpenInference Dynatrace: where should AI traces become gen_ai.*? OpenTelemetry Collector setup, a test run and DQL checks, on Dynatrace SaaS.
Ollama vLLM llama.cpp: which local engine should you choose for LLMs?
Ollama vLLM llama.cpp: compare these three local inference engines by use case, hardware and monitoring (metrics, health checks, traces).
RAG faithfulness: how to evaluate a RAG and where the metric stops
RAG faithfulness with Ragas and LlamaIndex: what the score measures, how to read it and why a faithful answer can still be wrong during on-call.
LLM gateway: track costs with LiteLLM and Helicone
LLM gateway: attribute token costs per team with LiteLLM (virtual keys, Prometheus) or Helicone, then export over OTLP to Dynatrace through a Collector.
Jan open source ChatGPT alternative: offline local AI
Jan open source ChatGPT alternative: offline local AI, an OpenAI-compatible server on localhost:1337, installation, limits and monitoring.
context-mode: agent context window, ELv2 licence and measurement
context-mode keeps tool output out of the agent context window. The ELv2 licence explained, and tokens saved measured with OpenTelemetry metrics.
Selecto AI Windows: an action bar for any selected text
Selecto AI Windows adds an action bar to selected text: search, translate, summarise and correct with local Ollama AI or your own API key.
AI engineering from scratch learning path: the repo curriculum
AI engineering from scratch learning path: 542 MIT-licensed lessons, three ways to start and an agent-trace workshop for SRE teams to adopt.