MNIST Data Classifier

In this example, we will be creating a keras DNN model to identify the images MNIST dataset as numerical digits.

We will take the following steps:

  1. Use an existing project we created for working with MNIST dataset.

  2. Create an experiment to track the training process where we will record the hyper-parameters, loss curve, epoch time, CPU stats etc.

  3. Evaluate the trained model against the test dataset

Pre-Requisites

We recommend running this example in a virtual environment for running this example. Also, we recommend using anaconda for machine learning.

  1. Please complete the setup of your machine.

  2. Install Pandas, Keras, Tensorflow

pip install pandas scikit-learn keras tensorflow

Example

import time

from keras.datasets import mnist
from keras.layers import Dense
from keras.models import Sequential
from keras.optimizers import RMSprop
from keras.utils import to_categorical

import markov
from markov.api.schemas.model_recording import SingleTagInferenceRecord

# Load an existing project using its id. You can find the list of projects here: `app.markovml.com/<workspace_id>/proj`
# This is an optional step, you can directly use the project id in experimentation and evaluation of your model
mnist_project = markov.Project.from_id(project_id="3z9Qku9rbVX3BS")

# Load your dataset, in this case we are using the keras library to load the dataset. You can load it in any way.
(x_train, y_train), (x_test, y_test) = mnist.load_data()

IMAGE_INPUT_SHAPE = 784

# Minor data pre-processing
# reshape, convert to float and normalize to send standard input into the DNN
x_train = x_train.reshape(-1, IMAGE_INPUT_SHAPE).astype("float32") / 255.0
x_test = x_test.reshape(-1, IMAGE_INPUT_SHAPE).astype("float32") / 255.0

num_classes = 10  # since there are 10 digits in which we are classifying the images to

# convert class vectors to binary class matrices
y_train = to_categorical(y_train, num_classes)
y_test = to_categorical(y_test, num_classes)


# build the model that will be used to classify the MNIST images
def _build_model_graph(input_shape=(IMAGE_INPUT_SHAPE,)):
    model = Sequential()
    model.add(Dense(512, activation="relu", input_shape=input_shape))
    model.add(Dense(512, activation="relu"))
    model.add(Dense(10, activation="softmax"))
    model.compile(
        loss="categorical_crossentropy", optimizer=RMSprop(), metrics=["accuracy"]
    )

    return model


MODEL_NAME = f"Classification of MNIST Dataset using Keras DNN {int(time.time())}"


# auto_record will automatically track the experiment - including its hyper-parameters, loss curve, epoch time etc.
markov.keras.auto_record(
    name=MODEL_NAME,
    notes="This experiment is used to track the training process of the Keras DNN used for classification of MNIST.",
    project_id=mnist_project.project_id,  # you can simply paste the project_id here as well
)

model = _build_model_graph()

# The training process will automatically be tracked as we used "auto_record" above.
model.fit(x_train, y_train, batch_size=128, epochs=5)


# Now let us evaluate this model against (x_test, y_test)
evaluation_recorder = markov.EvaluationRecorder(
    name=f"Evaluate {MODEL_NAME}",
    model_id=model.markov_model_id,
    project_id=mnist_project.project_id,
)

evaluation_recorder.register()

y_pred = model.predict(x_test)

urid = 1
for pred, actual in zip(y_pred, y_test):
    evaluation_record = SingleTagInferenceRecord(
        inferred=pred.argmax().item(),
        actual=actual.argmax().item(),
        score=pred.max().item(),
        urid=urid,
    )
    urid = urid + 1

    evaluation_recorder.add_record(evaluation_record)

outcome = evaluation_recorder.finish()

Last updated

Was this helpful?