Create an Evaluation Recorder
You can run a model evaluation against a registered model object. There are two ways to create a model object on Markov.
Note: The model object is a harness for the model artifact generated from a model-training process.
Using Experiment Recorder
When an experiment is run, the model object is automatically created and registered with MarkovML. You can get the model_id
of the associated model object by the model_id
property of the ExperimentRecorder instance.
Creating a Model when using auto_record
When an experiment is run using auto_record
, the model object is automatically created and registered with MarkovML.
For example, if t created an experiment using a keras
model by calling markov.keras.auto_record
. After you have called model.fit()
method, you can get the model_id
using model.markov_model_id
on the model
on which fit()
was called.
Creating New Model
You can also create the Model placeholder explicitly using SDK.
To run an evaluation against an existing model, you can get the model object as follows.
The evaluation dataset should be registered with markovML. This is required to create a lineage between the dataset and evaluations. This also helps in root cause analysis to plan future iterations.
Getting Existing Model for Evaluation
Evaluating a Model
After creating an evaluation recorder, you can proceed to input evaluation records. Further instructions can be found on the following page.
You can evaluate a model multiple times by supplying the same model_id
to your EvaluationRecorder
.
Last updated
Was this helpful?