Package com.rpl.agentorama

Interface AgentManager

All Superinterfaces:
AutoCloseable, Closeable, com.rpl.agentorama.impl.IFetchAgentClient
public interface AgentManager extends com.rpl.agentorama.impl.IFetchAgentClient, Closeable
Manager for interacting with deployed agents on a Rama cluster. The agent manager provides access to agent clients, dataset management, and evaluation capabilities for a specific module deployed on a cluster. Example: 

 AgentManager manager = AgentManager.create(cluster, "MyModule");
 Set<String> agentNames = manager.getAgentNames();
 AgentClient client = manager.getAgentClient("my-agent");
 String result = client.invoke("Hello world");

  • Method Details

    • create

      static AgentManager create(com.rpl.rama.cluster.ClusterManagerBase cluster, String moduleName)
      Creates an agent manager for managing and interacting with deployed agents on a Rama cluster.
      Parameters:
      cluster - the Rama cluster instance (IPC or remote cluster)
      moduleName - the name of the deployed module
      Returns:
      interface for managing agents and datasets

    • getAgentNames

      Set<String> getAgentNames()
      Gets the names of all available agents in the module.
      Returns:
      set of agent names available in the module

    • createDataset

      UUID createDataset(String name, String description, String inputJsonSchema, String outputJsonSchema)
      Creates a new dataset for agent testing and evaluation. Datasets are collections of input/output examples used for testing agent performance, running experiments, and regression testing.
      Parameters:
      name - the name of the dataset
      description - description of what the dataset contains
      inputJsonSchema - JSON schema for input validation
      outputJsonSchema - JSON schema for output validation
      Returns:
      UUID of the created dataset

    • setDatasetName

      void setDatasetName(UUID datasetId, String name)
      Updates the name of an existing dataset.
      Parameters:
      datasetId - UUID of the dataset
      name - new name for the dataset

    • setDatasetDescription

      void setDatasetDescription(UUID datasetId, String description)
      Updates the description of an existing dataset.
      Parameters:
      datasetId - UUID of the dataset
      description - new description for the dataset

    • destroyDataset

      void destroyDataset(UUID datasetId)
      Permanently deletes a dataset and all its examples.
      Parameters:
      datasetId - UUID of the dataset to delete

    • addDatasetExampleAsync

      CompletableFuture<Void> addDatasetExampleAsync(UUID datasetId, Object input, AddDatasetExampleOptions options)
      Asynchronously adds an example to a dataset.
      Parameters:
      datasetId - UUID of the dataset
      input - input data for the example
      options - configuration options for the example
      Returns:
      future that completes when the example is added

    • addDatasetExample

      UUID addDatasetExample(UUID datasetId, Object input, AddDatasetExampleOptions options)
      Adds an example to a dataset for testing and evaluation.
      Parameters:
      datasetId - UUID of the dataset
      input - input data for the example
      options - configuration options for the example
      Returns:
      UUID of the added example

    • setDatasetExampleInput

      void setDatasetExampleInput(UUID datasetId, String snapshotName, UUID exampleId, Object input)
      Updates the input data for a specific dataset example.
      Parameters:
      datasetId - UUID of the dataset
      snapshotName - name of the snapshot (or null for current)
      exampleId - UUID of the example
      input - new input data for the example

    • setDatasetExampleReferenceOutput

      void setDatasetExampleReferenceOutput(UUID datasetId, String snapshotName, UUID exampleId, Object referenceOutput)
      Updates the reference output for a specific dataset example.
      Parameters:
      datasetId - UUID of the dataset
      snapshotName - name of the snapshot (or null for current)
      exampleId - UUID of the example
      referenceOutput - new reference output for the example

    • removeDatasetExample

      void removeDatasetExample(UUID datasetId, String snapshotName, UUID exampleId)
      Removes a specific example from a dataset.
      Parameters:
      datasetId - UUID of the dataset
      snapshotName - name of the snapshot (or null for current)
      exampleId - UUID of the example to remove

    • addDatasetExampleTag

      void addDatasetExampleTag(UUID datasetId, String snapshotName, UUID exampleId, String tag)
      Adds a tag to a specific dataset example for categorization.
      Parameters:
      datasetId - UUID of the dataset
      snapshotName - name of the snapshot (or null for current)
      exampleId - UUID of the example
      tag - tag to add

    • removeDatasetExampleTag

      void removeDatasetExampleTag(UUID datasetId, String snapshotName, UUID exampleId, String tag)
      Removes a tag from a specific dataset example.
      Parameters:
      datasetId - UUID of the dataset
      snapshotName - name of the snapshot (or null for current)
      exampleId - UUID of the example
      tag - tag to remove

    • snapshotDataset

      void snapshotDataset(UUID datasetId, String fromSnapshotName, String toSnapshotName)
      Creates a snapshot of a dataset at its current state.
      Parameters:
      datasetId - UUID of the dataset
      fromSnapshotName - name of the source snapshot (or null for current)
      toSnapshotName - name for the new snapshot

    • removeDatasetSnapshot

      void removeDatasetSnapshot(UUID datasetId, String snapshotName)
      Removes a specific snapshot from a dataset.
      Parameters:
      datasetId - UUID of the dataset
      snapshotName - name of the snapshot to remove

    • searchDatasets

      Map<UUID,String> searchDatasets(String searchString, int limit)
      Searches for datasets by name or description.
      Parameters:
      searchString - string to search for in names and descriptions
      limit - maximum number of results to return
      Returns:
      map from dataset UUID to dataset name

    • createEvaluator

      void createEvaluator(String name, String builderName, Map params, String description, CreateEvaluatorOptions options)
      Creates an evaluator instance from a builder for measuring agent performance in experiments or actions.
      Parameters:
      name - name for the evaluator
      builderName - name of the evaluator builder (declared in topology or built-in)
      params - map of parameters for the evaluator. Parameters are a map from parameter name to parameter value, both strings.
      description - description of what the evaluator measures
      options - configuration options for the evaluator

    • removeEvaluator

      void removeEvaluator(String name)
      Removes an evaluator from the system.
      Parameters:
      name - name of the evaluator to remove

    • searchEvaluators

      Set<String> searchEvaluators(String searchString)
      Searches for evaluators by name or description.
      Parameters:
      searchString - string to search for in evaluator names
      Returns:
      set of matching evaluator names

    • tryEvaluator

      Map tryEvaluator(String name, Object input, Object referenceOutput, Object output)
      Tests an evaluator on a single sample input / reference output / output.
      Parameters:
      name - name of the evaluator
      input - input data for the evaluation
      referenceOutput - reference output for comparison
      output - actual output to evaluate
      Returns:
      result scores from score name to score value

    • tryComparativeEvaluator

      Map tryComparativeEvaluator(String name, Object input, Object referenceOutput, List<Object> outputs)
      Tests a comparative evaluator on multiple outputs.
      Parameters:
      name - name of the evaluator
      input - input data for the evaluation
      referenceOutput - reference output for comparison
      outputs - collection of actual outputs to compare
      Returns:
      comparative evaluation result, a map of score name to score value

    • trySummaryEvaluator

      Map trySummaryEvaluator(String name, List<ExampleRun> exampleRuns)
      Tests a summary evaluator on a collection of example runs.
      Parameters:
      name - name of the evaluator
      exampleRuns - collection of example runs created with ExampleRun.create()
      Returns:
      summary evaluation result with aggregate metrics, a map from score name to score value