Create Experiment Recorder

Markov uses Recorder objects to record data into the MarkovML backend. Recording data from a model training experiment with MarkovML is a three-step process.

1. Create an experiment recorder object.

2. Add experiment records to the recorder to send to the MarkovML backend.

3. Call finish on recorder object to signal to finish of a recording.

When you create an experiment recorder, you'll provide a name for the training session, the ID of the relevant Project, as well as any relevant hyper-parameter values used in the model training. You may also add any notes you'd like to save with the experiment. See the code example below for more details.

Before creating the experiment recorder, get a reference to a Project (to specify the project ID) and define the hyper-parameters used for your model training.

Follow the sample code below to create an experiment tracking recorder:

import markov

# Get the existing project by name or id
my_project = markov.Project.get_by_name(name="My first project")
# or my_project = markov.Project.get_by_id(project_id="PROJECT_ID")

# Define the model here
   # hyper_parameters = {"learning_rate":0.1, "n_input":100, ...}
   # model = torch.nn.Sequential()

# Alternatively, you can use a project's create_experiment_recorder method.
# In this case, the project_id argument will be inferred.
recorder = my_project.create_experiment_recorder(
    name = "Test_Experiment_Tracking_With_MarkovML",
    notes = "This is a test experiment",
    hyper_parameters ={
        "learning_rate": 0.1,
        "n_input": 100,
        "n_hidden": 50,
        "n_output": 10
    },

   # Optional Key/Value pair to store values for custom variables with MarkovML
   # for this experiment. For example
   #     meta_data = {
   #             "exp_code": "AGI",
   #             "config":{
   #                         "feature_store": 1.2, 
   #                          "lexicon":"/path/secret_store/table.id"
   #                      }
   #          }
    meta_data={"KEY": "VALUE_PAIR"} 
)

# Register the experiment recorder with the MarkovML backend. Only a registered
# experiment recorder can be used to add records.
recorder.register()

Note: The hyper_parameters field for the recorder can take any key-value pairs you want to register with the recorder as a hyper-parameter.

When an experiment recorder is instantiated, two new resources are created in MarkovML:

1. An Experiment

2. A Model object.

This Model object is a placeholder to store a pointer to the artifact generated from the training session in the future. After creating the recorder, we can send the experimentation data to MarkovML. Please refer to the next page for further instructions.