Add Records to an Experiment Recorder

In this article, we will go over recording experiment data through the experiment recorder during the model training (experiment) step. The instructions to create an experiment recorder are here.

An ExperimentRecorder has the following key methods:

• start(): start the model training and should be called just before the model training code begins executing.

• stop(): end the model training and should be called right after the model training code has finished.

• add_record({"key": value}): The API used to record metrics with MarkovML. Here key could be any string and value could be any numeric value. The add_record should be called the training loop to record metrics like loss and accuracy. These recorded metrics will be available as charts in the MarkovML app.

The recorder instance can also be utilized as a context manager using the with statement to avoid explicit calls to the start() and stop() methods.

Follow the sample code pattern below to add records to the experiment tracking recorder.

import markov

# Your model definition 
# model = ...

# create recorder
recorder = markov.ExperimentRecorder(
    name="MarkovML experiment #1"
)
# The register method registers this recorder with the MarkovML backend. 
# You can record data with the backend only through a registered recorder.
recorder.register()

with recorder:
    for epoch in range(500):
        pred = model(x)  
        # Calculate and record loss
        loss = loss_function(pred, y)
        recorder.add_record({"loss": loss})
        
        # calculate and record accuracy
        accuracy = accuracy_function(pred, y)
        recorder.add_record({"accuracy": accuracy})
        
        # REST OF THE MODEL TRAINING CODE GOES HERE

Markov has easy integrations with many popular machine-learning frameworks. Please refer to the integration section on the next page for details.