Image Classification with Pre-Trained Models in Keras

In other posts, we explained how to apply Object Detection in Tensorflow and Object Detection using YOLO. Today we will provide a practical example of how we can use “Pre-Trained” ImageNet models using Keras for Object Detection. For this example, we will consider the Xception model but you can use anyone from the list here. The table below shows the size of the pre-trained models, their performance and their complexity in terms of parameters of the Convolutional Neural Network Architecture.

Image Classification with Xception

from keras.applications.xception import Xception
from keras.preprocessing import image
from keras.applications.xception import preprocess_input, decode_predictions
import numpy as np

import PIL
from PIL import Image
import requests
from io import BytesIO


# load the model
model = Xception(weights='imagenet', include_top=True)


# chose the URL image that you want
URL = "https://images.unsplash.com/photo-1529429617124-95b109e86bb8?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=500&q=60"


# get the image
response = requests.get(URL)
img = Image.open(BytesIO(response.content))
# resize the image according to each model (see documentation of each model)
img = img.resize((299,299))

##############################################
# if you want to read the image from your PC 
#############################################
# img_path = 'myimage.jpg'
# img = image.load_img(img_path, target_size=(299, 299))
#############################################



# convert to numpy array
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

features = model.predict(x)

# return the top 20 detected objects
label = decode_predictions(features, top=20)
label

The image that we chose is the following:

And the labels that we get are:

[[('n02111889', 'Samoyed', 0.781301),
  ('n02114548', 'white_wolf', 0.124226466),
  ('n02120079', 'Arctic_fox', 0.005746077),
  ('n02111500', 'Great_Pyrenees', 0.0033085805),
  ('n02104029', 'kuvasz', 0.00311469),
  ('n02112137', 'chow', 0.0028483241),
  ('n02112018', 'Pomeranian', 0.0028328209),
  ('n02109961', 'Eskimo_dog', 0.002488005),
  ('n02106030', 'collie', 0.0016986788),
  ('n02085782', 'Japanese_spaniel', 0.0008878598),
  ('n02114367', 'timber_wolf', 0.0007279106),
  ('n02134084', 'ice_bear', 0.0007163896),
  ('n02112350', 'keeshond', 0.00068674894),
  ('n02098286', 'West_Highland_white_terrier', 0.00066097657),
  ('n02123394', 'Persian_cat', 0.0005676047),
  ('n02106166', 'Border_collie', 0.0005447453),
  ('n02105056', 'groenendael', 0.00047748425),
  ('n02099601', 'golden_retriever', 0.00039739575),
  ('n02094114', 'Norfolk_terrier', 0.0003679685),
  ('n02085936', 'Maltese_dog', 0.00032893682)]]

We chose to return the top 20 most likely detected objects. The tuple has three elements, the first one is referred to the ObjectID, the second one to the Object Label and the third one to the Probability. So let’s say that we want to get all the objects which have a probability higher than 5%. Then we can type:

[tpl[1:3] for tpl in label[0] if tpl[2]>0.05]

[('Samoyed', 0.781301), ('white_wolf', 0.124226466)]

As we can see, the Xception model detected as the most likely objects in the image the “Samoyed” and the “white_wolf“. The model did a good job since the breed that we chose was Samoyed! Feel free to test it with other objects. Just notice that it tries to detect one object. For detecting many objects in one image we will discuss in another post!

Note: The pre-trained models in Keras try to find out one object per image. Thus, it is like a classification algorithm where it assigns for every object one probability and all of them ad up to 1. That being said, it cannot detect multiple objects in one image.