Okareo

Open-source LLM observability, evaluation, and agent testing.

An open-source platform for monitoring, evaluating, and testing LLM and agent applications. LangWatch captures traces, runs evaluations and simulations, and surfaces quality and cost metrics in production. Offered as managed cloud or fully self-hosted for teams with strict data-residency needs.

Simulate, evaluate, and observe AI agents end-to-end.

An end-to-end platform for testing and monitoring AI agents across their lifecycle. It combines a prompt experimentation IDE, agent simulation across scenarios and personas, offline and online evaluations with custom metrics, and production observability with tracing and alerts. Aimed at teams shipping reliable agentic and RAG systems.

Open-source LLM eval CLI. Rubric scoring + golden sets.

YAML-driven eval harness. Pair a prompt with a goldset, define rubrics, run across multiple models in CI. Strong for catching prompt regressions before they hit production.

The AI quality platform from the team behind DeepEval.

Confident AI is the hosted platform built on top of DeepEval, the open-source LLM evaluation framework. It adds dataset and test management, research-backed metrics, production tracing and monitoring, adversarial red teaming, and governance dashboards so teams can benchmark, observe, and safeguard LLM apps across the dev-to-prod loop. Python and TypeScript SDKs plug into CI and OpenTelemetry, with managed cloud and enterprise self-hosting.