End-to-end platform for building and perfecting LLM agents on the JVM
Orchestration
Define agents as simple graphs of Java or Clojure functions that execute in parallel. Built-in durable storage, first-class streaming, and human-in-the-loop APIs let you focus on agent logic, not infrastructure. Deploy on Rama clusters that scale from one node to thousands.
- Graph-based parallel execution
- Integrated durable storage
- First-class streaming to clients
- Human-in-the-loop at any point
- Scales horizontally on Rama clusters
Evaluation
Iterate with data, not guesswork. Create datasets and run experiments with automated evaluators and human feedback. Track latency, token usage, and quality metrics with built-in telemetry. Production actions sample and evaluate live traffic, feeding results back into your datasets.
- Datasets with immutable snapshots
- Experiments with side-by-side comparison
- LLM-as-judge and custom evaluators
- Structured human feedback queues
- Time-series telemetry for all metrics
The seamless evaluation loop
Agent-o-rama unifies orchestration and evaluation into a single continuous cycle. Production data flows back into datasets automatically, so every deployed agent is also being tested.
Define
Build agents as graphs of functions
Trace
Automatic capture of every execution detail
Evaluate
Score with automated evaluators and human feedback
Iterate
Fork traces and test changes instantly
Deploy
One-click deployment to your infrastructure
Monitor
Track metrics with telemetry, auto-evaluate and enrich datasets
Key capabilities
Graph-based agents
Define agents as graphs of pure Java or Clojure functions that execute in parallel.
Comprehensive tracing
Every node emit, model call, tool call, token count, and timing is captured automatically. Fork any trace to test changes.
Datasets & experiments
Build datasets from traces or production traffic. Run experiments to determine the best tradeoff between agent quality, latency, and cost.
Automated & human evaluation
Score outputs with LLM-as-judge and custom evaluators. Collect structured and unstructured human feedback through review queues.
Production monitoring
Time-series telemetry for latency, token usage, and quality scores. Actions automatically evaluate runs, enrich datasets, and trigger webhooks.
Integrates with anything
Agents are pure Java or Clojure functions. Call any database, API, or library directly. Include all remote calls in traces.
OPEN SOURCE ON GITHUB
Agent-o-rama is free for clusters up to two nodes. Get started in minutes with the quickstart guide.
QuickstartJoin our Discord
Join the Rama Discord community to ask questions and connect with other developers
Join Discord