Hindsight AI agent memory: install and monitor it

Hindsight AI agent memory, open source: README install steps, Prometheus metrics, OpenTelemetry traces, token cost and privacy points to supervise.
Hindsight mémoire agents IA : installer et superviser

Hindsight AI agent memory: the vectorize-io/hindsight repository offers a memory system you run alongside an agent so it retains what it learns from one session to the next. Its tagline: “Agent Memory That Learns”. It showed about 50,500 stars in mid-October 2026, under the MIT licence, and version 1.0.0 was released on 13 October 2026.

Agent memory changes the nature of the service you supervise. It stores data (often personal), it calls a language model on every write and every consolidation, and it sits in the agent’s response path. Latency, token cost, recall quality and privacy all become operational topics. This article presents the project from its README, the installation, then supervision: Prometheus metrics, OpenTelemetry traces and useful alerts.

The information comes from the project’s README and documentation. span.observer did not install Hindsight: the diagrams below come from the repository.

Hindsight AI agent memory: what the repository says

  • Publisher: Vectorize AI, Inc. (copyright line of the MIT LICENSE file). Hindsight™ is a trademark; the same company offers a hosted service, Hindsight Cloud, billed by usage according to the README.
  • Technology: a Python server (PyPI package hindsight-api), Python, Node.js and Go clients plus a CLI, PostgreSQL storage with pgvector (or Oracle AI Database 23ai).
  • Principle: three operations. Retain stores information (a language model extracts facts, entities, relationships and dates), recall finds it with four strategies in parallel (semantic, BM25 keyword, graph, temporal), reflect produces a deeper analysis of memories.
  • Organisation: memories live in “banks”; types are world facts, experiences, consolidated observations and mental models.
  • Integrations: the README lists more than sixty (coding agents, frameworks, no-code tools) and a built-in MCP server per bank, at http://localhost:8888/mcp/{bank_id}/.
Diagram from the Hindsight README: an agent retains, recalls and reflects using memory networks made of world facts, experiences, observations and opinions
Source : github.com

Installing Hindsight from the README

Step 1 — Start the server

export OPENAI_API_KEY=sk-xxx

docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
  -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
  -v hindsight-data:/home/hindsight/.pg0 \
  ghcr.io/vectorize-io/hindsight:latest

The API listens on http://localhost:8888 and the UI on http://localhost:9999. The key above is a placeholder. Hindsight works with more than 25 model providers through HINDSIGHT_API_LLM_PROVIDER, including local models (Ollama, LM Studio, llama.cpp). Without Docker: pip install hindsight-api, then hindsight-api. The README also offers a Helm chart for Kubernetes.

Step 2 — Connect a client

pip install hindsight-client -U
from hindsight_client import Hindsight

client = Hindsight(base_url="http://localhost:8888")

client.retain(bank_id="my-bank", content="Alice works at Google as a software engineer")
client.recall(bank_id="my-bank", query="What does Alice do?")
client.reflect(bank_id="my-bank", query="Tell me about Alice")

For an existing agent, the README describes a two-line “LLM Wrapper” (pip install hindsight-litellm, then wrap_openai): memories are recalled before the model call and the conversation is retained afterwards.

Step 3 — Isolate memories per user

The README illustrates isolation through metadata (a user identifier) and the full flow, from retaining to sending memories to the model.

Six-step guide from the Hindsight README for per-user memories: install the SDK, prepare metadata, retain, form mental models, recall, send to the model
Source : github.com

Supervising an agent’s memory

Hindsight exposes what operations needs: Prometheus metrics on /metrics, OpenTelemetry traces and two health probes. Memory cost is read the same way as the agent’s own cost.

Traces and metrics

The monitoring documentation describes tracing that follows the GenAI semantic conventions (v1.37 and later). Enable it with two variables, then send to any OTLP backend:

Excerpt from the Hindsight monitoring documentation: HINDSIGHT_API_OTEL_TRACES_ENABLED and HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT variables, a /metrics call and two PromQL queries
  • Operation spans: hindsight.retain, hindsight.recall (with its embedding, retrieval, fusion and reranking sub-steps), hindsight.reflect, hindsight.consolidation.
  • Model spans: attributes gen_ai.provider.name, gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, plus hindsight.scope (memory, reflect, consolidation…).
  • Model metrics: hindsight.llm.calls.total, hindsight.llm.duration, hindsight.llm.tokens.input and hindsight.llm.tokens.output, broken down by provider, model, scope and success. On the Prometheus side, dots become underscores (hindsight_llm_tokens_input_total).
  • Operations: hindsight.operation.duration and hindsight.operation.total per operation (retain, recall, reflect).

Dedicated workers export their own spans (default service name hindsight-worker), and the documentation places consolidation, background retain and mental-model refresh there, which is most of the long-running work and token spend; those spans form traces separate from the originating request. The bank_id and tenant labels are off by default to limit cardinality; HINDSIGHT_API_METRICS_INCLUDE_BANK_ID=true turns them on.

Useful alerts

  • Tokens by scope: sum hindsight_llm_tokens_input_total and …_output_total by scope: a consolidation that balloons costs money without the agent answering faster.
  • Documents with no memory: the documentation gives the ratio of hindsight_retain_documents_total{outcome="no_facts"} to the total; a rise signals a retain that succeeds but produces nothing recallable.
  • Consolidation failures: hindsight_consolidation_batch_failures_total{failure_class="fail_fast"} shows the model not producing JSON that matches the schema.
  • Recall latency: recall runs in the agent’s response path; track the 95th percentile of hindsight_operation_duration for recall.

Health

Two probes exist: /health/live (no database access, for liveness) and /health/ready or /health (with a SELECT 1, for readiness). The documentation warns: never point a liveness probe at a dependency, or a slow database restarts every pod.

How to check that it works

  • curl http://localhost:8888/metrics returns hindsight_* series after a first retain.
  • With tracing on, a recall operation appears with its sub-spans, and model spans carry gen_ai.usage.input_tokens.
  • A document that was added then recalled comes back with its content; if not, check the no_facts ratio.

Limits and open points

  • Performance claims. The README says Hindsight is “the most accurate agent memory system ever tested” on LongMemEval, with an independent reproduction, and that other scores are self-reported by vendors. These are the project’s statements: span.observer did not reproduce them.
  • Brand-new 1.0.0 (13 October 2026): expect adjustments and read the release notes before upgrading.
  • Sensitive data. A memory holds what your users said. The documentation states that model spans carry full prompts and completions as events: decide what your trace backend may keep. Banks isolate users, but erasure and retention are yours to define.
  • Model dependency. Every retain calls a language model: cost, latency and extraction quality depend on your choice, see choosing an LLM.
  • Commercial offer. Hindsight Cloud sits next to the MIT code: check what the self-hosted version leaves out.

Best practices

  • One bank per user or tenant, never a shared bank for private data.
  • Use a local model or a contractual provider for sensitive memory: see the self-hosted open-source AI stack.
  • Separate workers from the API and name their services in the trace backend.
  • Alert on consolidation tokens before you discover the invoice.
  • Cap context size and recalled memory; the article on agent context windows covers the topic.

Frequently asked questions

Does Hindsight replace RAG?

The project claims to overcome the limits of RAG and knowledge graphs; that is its own positioning. In practice the two complement each other: RAG queries documents, agent memory keeps what the agent has learned.

Do I need an external model provider?

No: the README lists Ollama, LM Studio and llama.cpp among supported providers. A provider key is only required for hosted models.

Can I observe Hindsight without Grafana?

Yes. Metrics are in Prometheus format and traces are OTLP; the documentation names Langfuse, OpenLIT, Datadog, New Relic and Honeycomb as possible destinations.

Conclusion

Hindsight gives agents persistent memory and equips it for operations: metrics, GenAI traces, health probes. Accuracy figures remain the project’s claims; what you can measure yourself is recall latency, token spend and the share of documents with no memory. For an agent that acts, see also OpenClaw.

span.observer is an independent community blog, not affiliated with Dynatrace LLC or the projects mentioned. Trademarks belong to their owners. Tools change quickly: check commands and options in their official documentation (hindsight.vectorize.io).

Total
0
Shares
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
Paperclip orchestration agents IA : installer et superviser

Paperclip AI agent orchestration: install and monitor

Next Post
NVIDIA OpenShell runtime agents IA : isoler et auditer

NVIDIA OpenShell agent runtime: isolate and audit AI agents

Related Posts