How to save and load a model with Keras

In this article, we’ll look at how we can save a model that we have trained in Keras.

First, we’ll create a very simple CNN model using Keras and train that model on the Fashion MNIST data set. Then I will show you how to save and subsequently restore that trained model and perform the prediction.

I understand that saving a model for a very basic problem like fashion MNIST might sound trivial.

However, think of a situation where we are training a very complex model which takes a tremendous amount of time, like hours, or even days, to train.

In such cases, it is often useful to save the model. So, in the future, if we need to make predictions, we can simply load the trained model and make our predictions instead of training our model again from scratch.

In Keras, models are saved in the following three formats.

Let’s see how to save the model in each of these formats.

HDF5:

Generally, the models are saved in this format. HDF5 is short for the Hierarchical Data Format. The number five indicates the version.

The HDF format will store our entire model. It saves all the information about our model, including the architecture, model weights, trained parameters, optimizer details etcetera.

In Keras, saving a model in the HDF5 format is quite simple. Keras has a save() function that allows us to easily save the model in this format.

model.save(‘model.h5’)

model.save(‘model.h5’)

h5 is the extension used to save the model in HDF format.

YAML OR JSON:

Instead of saving the entire model Keras also gives us the flexibility to save the models architecture and weights separately.

We can use either of these two formats to save only the model’s architecture without the weights, parameters, loss, or optimizer settings.

We can use the following functions to save the model in JSON or YAML format.

model.to_json() # to save model as json model.to_yaml() # to save model as yaml

model.to_json() # to save model as json

model.to_yaml() # to save model as yaml

Since these formats save only the architecture we need to save the model’s weight separately.

Weights can be stored in the h5 format. To save the weights we can use the save_weights() method.

model.save_weights(‘weight.h5’)

model.save_weights(‘weight.h5’)

Now let’s do some coding.

Firstly, we’ll import all the necessary packages and the dataset. We’ll use the fashion MNIST dataset.

import keras import numpy as np from keras.models import Sequential from keras.layers import Dense, Dropout, Flatten from keras.layers import Conv2D, MaxPooling2D from keras import backend as K from keras.datasets import fashion_mnist from keras.utils import np_utils

import keras

import numpy as np

from keras.models import Sequential

from keras.layers import Dense, Dropout, Flatten

from keras.layers import Conv2D, MaxPooling2D

from keras import backend as K

from keras.datasets import fashion_mnist

from keras.utils import np_utils

Next we’ll create our CNN model. The below code snippet is taken from Keras Github Repo.

batch_size = 128 num_classes = 10 epochs = 50 # input image dimensions img_rows, img_cols = 28, 28 # the data, split between train and test sets (x_train, y_train), (x_test, y_test) = fashion_mnist.load_data() if K.image_data_format() == ‘channels_first’: x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols) x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols) input_shape = (1, img_rows, img_cols) else: x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1) x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1) input_shape = (img_rows, img_cols, 1) x_train = x_train.astype(‘float32’) x_test = x_test.astype(‘float32’) x_train /= 255 x_test /= 255 print(‘x_train shape:’, x_train.shape) print(x_train.shape[0], ‘train samples’) print(x_test.shape[0], ‘test samples’) # convert class vectors to binary class matrices y_train = keras.utils.to_categorical(y_train, num_classes) y_test = keras.utils.to_categorical(y_test, num_classes) #Building our CNN model = Sequential() model.add(Conv2D(32, kernel_size=(3, 3), activation=’relu’, input_shape=input_shape)) model.add(Conv2D(64, (3, 3), activation=’relu’)) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(128, activation=’relu’)) model.add(Dropout(0.5)) model.add(Dense(num_classes, activation=’softmax’)) #compile the model model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adadelta(), metrics=[‘accuracy’])

batch_size = 128

num_classes = 10

epochs = 50

# input image dimensions

img_rows, img_cols = 28, 28

# the data, split between train and test sets

(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()

if K.image_data_format() == ‘channels_first’:

x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)

x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)

input_shape = (1, img_rows, img_cols)

else:

x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)

x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)

input_shape = (img_rows, img_cols, 1)

x_train = x_train.astype(‘float32’)

x_test = x_test.astype(‘float32’)

x_train /= 255

x_test /= 255

print(‘x_train shape:’, x_train.shape)

print(x_train.shape[0], ‘train samples’)

print(x_test.shape[0], ‘test samples’)

# convert class vectors to binary class matrices

y_train = keras.utils.to_categorical(y_train, num_classes)

y_test = keras.utils.to_categorical(y_test, num_classes)

#Building our CNN

model = Sequential()

model.add(Conv2D(32, kernel_size=(3, 3),

activation=‘relu’,

input_shape=input_shape))

model.add(Conv2D(64, (3, 3), activation=‘relu’))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

model.add(Flatten())

model.add(Dense(128, activation=‘relu’))

model.add(Dropout(0.5))

model.add(Dense(num_classes, activation=‘softmax’))

#compile the model

model.compile(loss=keras.losses.categorical_crossentropy,

optimizer=keras.optimizers.Adadelta(),

metrics=[‘accuracy’])

Finally, we can train our model using the model.fit() function.

history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_split=0.2, )

history = model.fit(x_train, y_train,

batch_size=batch_size,

epochs=epochs,

verbose=1,

validation_split=0.2,

)

Since we now have our trained model, let’s see how to save our model in the HDF5 format.

As we have already seen, it is super simple. We just need to call model.save() method.

model.save(‘Fashion_Mnist_CNN.h5’)

model.save(‘Fashion_Mnist_CNN.h5’)

That’s it. The model will be saved in your current working directory.

In most cases, we will not load a saved model in the same session. However, for demonstration purposes, let’s load our model.

from keras.models import load_model restored_model = load_model(‘/content/Fashion_Mnist_CNN.h5’)

from keras.models import load_model

restored_model = load_model(‘/content/Fashion_Mnist_CNN.h5’)

Now we can use our restored model to make predictions.

index = np.random.randint(0, len(x_test)-1) # randomly choose a sample from the test set sample = np.array([x_test[index]]) prediction = restored_model.predict(sample) print(‘Sample Number:’,index,’\n’,’\nPredicted Class:’,np.argmax([prediction[0]]), ‘Actual Class:’,np.argmax(y_test[index]))

index = np.random.randint(0, len(x_test)–1) # randomly choose a sample from the test set

sample = np.array([x_test[index]])

prediction = restored_model.predict(sample)

print(‘Sample Number:’,index,‘\n’,‘\nPredicted Class:’,np.argmax([prediction[0]]), ‘Actual Class:’,np.argmax(y_test[index]))

Output:- Sample Number: 5902 Predicted Class: 5 Actual Class: 5

Output:-

Sample Number: 5902

Predicted Class: 5 Actual Class: 5

Similarly, let’s look at how to save the model in JSON and YAML formats.

Save model in JSON and YAML formats:

Firstly we’ll see how to save the model in json format. The following code snippet will save our model in the json format.

json_string = model.to_json() #save our model as json with open(‘model.json’,’w’) as json_file: json_file.write(json_string)

json_string = model.to_json() #save our model as json

with open(‘model.json’,‘w’) as json_file:

json_file.write(json_string)

The model is converted into a JSON string on the first line, and afterward, on the second and third line, the JSON string is stored into a file named model.json.

Since JSON saves only the architecture, we need to save the model’s weight separately.

Weights can be stored in the h5 format. The save_weights () method can be used to save the weights.

#saving model weights model.save_weights(‘weights.h5’)

#saving model weights

model.save_weights(‘weights.h5’)

Since we now have our weights and architecture, let’s reconstruct our model from the JSON file.

from keras.models import model_from_yaml with open(‘/content/model.json’, ‘r’) as json_file: json_string = json_file.read() json_file.close() json_model = model_from_json(json_string)

from keras.models import model_from_yaml

with open(‘/content/model.json’, ‘r’) as json_file:

json_string = json_file.read()

json_file.close()

json_model = model_from_json(json_string)

The code above is pretty self-explanatory. Firstly, we are importing the class model_from_json.

On the second and third lines, we open the file and read its contents.

Finally, we pass the contents of the json_string to the model_from_json to reconstruct the model.

Now, if we need to make predictions, we need to load the weights and compile our model. To load the weights into our model, we can use the load_weights () function.

# load the weights json_model.load_weights(‘/content/weights.h5’) # compile the model new_model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adadelta(), metrics=[‘accuracy’])

# load the weights

json_model.load_weights(‘/content/weights.h5’)

# compile the model

new_model.compile(loss=keras.losses.categorical_crossentropy,

optimizer=keras.optimizers.Adadelta(),

metrics=[‘accuracy’])

We can now make predictions with our model.

index = np.random.randint(0, len(x_test)-1) # randomly choose a sample from the test set sample = np.array([x_test[index]]) prediction = json_model.predict(sample) print(‘Sample Number:’,index,’\n’,’\nPredicted Class:’,np.argmax([prediction[0]]), ‘Actual Class:’,np.argmax(y_test[index]))

index = np.random.randint(0, len(x_test)–1) # randomly choose a sample from the test set

sample = np.array([x_test[index]])

prediction = json_model.predict(sample)

print(‘Sample Number:’,index,‘\n’,‘\nPredicted Class:’,np.argmax([prediction[0]]), ‘Actual Class:’,np.argmax(y_test[index]))

OUTPUT:- Sample Number: 5645 Predicted Class: 5 Actual Class: 5

OUTPUT:-

Sample Number: 5645

Predicted Class: 5 Actual Class: 5

Saving a model in YAML is very similar to saving it in JSON. We just need to make a couple of changes.

Replace model.to_json with model.to_yaml and import model_from_yaml instead of model_from_json.

# saving the model yaml_model = model.to_yaml() with open(‘model.yaml’,’w’) as yaml_file: yaml_file.write(yaml_model) # loading the saved model from keras.models import model_from_yaml # model reconstruction from yaml with open(‘/content/model.yaml’,’r’) as yaml_file: yaml_string = yaml_file.read() yaml_model = model_from_yaml(yaml_string) #load weights yaml_model.load_weights(‘/content/weights.h5’) #compile the model yaml_model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adadelta(), metrics=[‘accuracy’]) # make predictions import numpy as np index = np.random.randint(0, len(x_test)-1) sample = np.array([x_test[index]]) prediction = yaml_model.predict(sample) print(‘Sample Number:’,index,’\n’,’\nPredicted Class:’,np.argmax([prediction[0]]), ‘Actual Class:’,np.argmax(y_test[index]))

# saving the model

yaml_model = model.to_yaml()

with open(‘model.yaml’,‘w’) as yaml_file:

yaml_file.write(yaml_model)

# loading the saved model

from keras.models import model_from_yaml

# model reconstruction from yaml

with open(‘/content/model.yaml’,‘r’) as yaml_file:

yaml_string = yaml_file.read()

yaml_model = model_from_yaml(yaml_string)

#load weights

yaml_model.load_weights(‘/content/weights.h5’)

#compile the model

yaml_model.compile(loss=keras.losses.categorical_crossentropy,

optimizer=keras.optimizers.Adadelta(),

metrics=[‘accuracy’])

# make predictions

import numpy as np

index = np.random.randint(0, len(x_test)–1)

sample = np.array([x_test[index]])

prediction = yaml_model.predict(sample)

print(‘Sample Number:’,index,‘\n’,‘\nPredicted Class:’,np.argmax([prediction[0]]), ‘Actual Class:’,np.argmax(y_test[index]))

There is also another method to save the model in Keras. It is by using the Callbacks.

Save model in Keras using Callbacks:

Keras callbacks are functions that are executed during the training process. According to Keras Documentation, A callback is a set of functions to be applied at given stages of the training procedure.

I have written an article explaining some of the commonly used callbacks. You can read the article Keras Callbacks to learn more about callbacks.

Now for this post let’s see how to save a model using Model Checkpoint callback.

By using model checkpoint callback, we can save our model at regular intervals.

The model checkpoint callback saves the weights of the model along with the structure once we made progress, like hitting a new validation accuracy or loss. We could also save the model for every epoch.

The signature of the callback is as follows

ModelCheckpoint(filepath, monitor=”val_loss”, verbose=0, save_best_only=False, save_weights_only=False, mode=”auto”, period=1)

ModelCheckpoint(filepath, monitor=‘val_loss’, verbose=0, save_best_only=False, save_weights_only=False, mode=‘auto’, period=1)

monitor: quantity to be monitored
save_best_only: if TRUE saves only the best model according to the quantity monitored
Save_weights_only: if TRUE only the weights of the model are saved
mode: one of {auto, min, max}. In min mode, training will stop when the quantity monitored has stopped decreasing; in max mode, it will stop when the quantity monitored has stopped increasing; auto mode, the direction is automatically inferred from the name of the monitored quantity.
period: after how many epochs you need to checkpoint the model

The code is same up until the model_compile() part. Add the following after the model.compile().

# import the callback from keras.callbacks import ModelCheckpoint # create the callback model_ckpt = ModelCheckpoint(monitor=”val_loss”, save_best_only=True, verbose=1, mode=”auto”,filepath=filepath) # pass the callback to the fit() method history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_split=0.2, callbacks=[model_ckpt])

# import the callback

from keras.callbacks import ModelCheckpoint

# create the callback

model_ckpt = ModelCheckpoint(monitor=‘val_loss’, save_best_only=True, verbose=1, mode=‘auto’,filepath=filepath)

# pass the callback to the fit() method

history = model.fit(x_train, y_train,

batch_size=batch_size,

epochs=epochs,

verbose=1,

validation_split=0.2,

callbacks=[model_ckpt])

In the above code snippet, we have created the Model Checkpoint callback and passed it to the fit function. It will save our model in the h5 format every time our validation loss improves.

Summary:

In this article, we looked at different ways to save a model and reconstruct it again in Keras.

Firstly, we saw how to save an entire model, including architecture, parameters trained, weights, loss, and optimizer states, etcetera.

Second, we discussed the JSON and YAML formats, which only store the architecture of the model.

Finally, we used the Keras Callback Model Checkpoint to save the model at regular intervals whenever we hit a better validation loss.

Visit this Github Repo to get the complete code.

Source link