Browsing Tag
Generative AI
11 posts
LLM-as-judge in production: keeping the budget under control
LLM-as-judge in production: sample, target, calibrate the judge on human examples and monitor its cost to keep the evaluation budget under control.
Cloudflare security audit skill: agent-led security audits and their limits
Cloudflare security audit skill: what the repository documents, how a skill is installed, its limits (false positives, scope) and how to monitor it.
ArtCraft Craft suite open source creative tools: overview
ArtCraft Craft suite open source creative tools: the AI platform and its Rust apps, MIT or Apache-2.0 licence, shared libraries and limits.
Davis CoPilot: querying your data in natural language, limits included
Davis CoPilot in practice: enable the AI, ask a precise question, review the generated DQL, block sensitive data and keep query consumption in check.
Self-hosted open source AI stack: comparing the building blocks of a self-hosted AI infrastructure
Compare the building blocks of a self-hosted open source AI stack: Ollama, vLLM, Open WebUI, LlamaIndex, CrewAI, Dify, n8n, Langfuse and typical setups.
Dify + n8n: orchestrating open source AI workflows, with ops use cases
Dify + n8n to automate alert triage, runbooks and post-mortems: key differences, Docker deployment, a local model and monitoring of every workflow.
LlamaIndex RAG on your internal documentation, and how to measure its quality
Build a self-hosted LlamaIndex RAG on your runbooks and procedures, then measure its quality: hit rate, MRR, faithfulness and relevancy of the answers.
Self-hosted Open WebUI: an internal ChatGPT in front of Ollama or an OpenAI-compatible API
Deploy self-hosted Open WebUI with Docker in front of Ollama or vLLM, secure it for a team, then trace every chat with OpenTelemetry through to the model.
Ollama local LLM: running models and monitoring them (metrics, logs, traces)
Ollama local LLM guide: install Ollama, serve a first model and monitor it with API probes, logs, GPU metrics and OpenTelemetry traces in Dynatrace.
How to measure AI ROI: moving from technical metrics to business outcomes with business observability
Measure AI ROI through the business process: baseline, cohorts, the real cost of tokens and conversion into euros with business observability in practice.