We will provide an example of how you can get the image labels using the AWS Rekognition. If you are not familiar with boto3, I would recommend having a look at the Basic Introduction to Boto3.
Object Detection with Rekognition using the AWS Console
You can start experimenting with the Rekognition on the AWS Console. Let’s have a look at the example that they provided. Notice that you can upload your own image as well.
As we can see from the screenshot above many objects returned as well as their corresponding confidence.
Object Detection with Rekognition using the Boto3
We can also get the image labels using Boto3. Let’s see how we can do it. For this example, we will use the same image as above. There are two ways to get the images, one is from the S3 and the other is from local files. We will show both ways.
Images on S3
I have created a bucket called 20201021-example-rekognition where I have uploaded the skateboard_thumb.jpg image. Let’s assume that I want to get a list of images labels as well as of their parents.
import boto3
client=boto3.client('rekognition')
# My bucket
bucket = '20201021-example-rekognition'
# My photo
photo = 'skateboard_thumb.jpg'
response = client.detect_labels(Image={'S3Object':{'Bucket':bucket,'Name':photo}},
MaxLabels=50)
# get a list of labels
label_lst = []
for label in response['Labels']:
label_lst.append(label['Name'])
# get a list of parents
parent_lst = []
for label in response['Labels']:
for parents in label['Parents']:
parent_lst.append(parents['Name'])
Let’s see the label_lst
['Town',
'Road',
'Urban',
'Street',
'Building',
'City',
'Human',
'Person',
'Pedestrian',
'Vehicle',
'Automobile',
'Transportation',
'Car',
'Downtown',
'Path',
'Neighborhood',
'Asphalt',
'Tarmac',
'High Rise',
'Alleyway',
'Alley',
'Apparel',
'Clothing',
'Photo',
'Photography',
'Architecture',
'Parking',
'Parking Lot',
'Face',
'Sedan',
'People',
'Selfie',
'Portrait',
'Apartment Building',
'Intersection']Let’s see also the parent_lst
['Urban',
'Building',
'City',
'Road',
'Urban',
'Building',
'Urban',
'Building',
'Person',
'Transportation',
'Vehicle',
'Transportation',
'Vehicle',
'Transportation',
'City',
'Urban',
'Building',
'Urban',
'Building',
'City',
'Urban',
'Building',
'Street',
'City',
'Road',
'Urban',
'Building',
'Street',
'City',
'Road',
'Urban',
'Building',
'Person',
'Person',
'Building',
'Car',
'Vehicle',
'Transportation',
'Car',
'Vehicle',
'Transportation',
'Person',
'Car',
'Vehicle',
'Transportation',
'Person',
'Portrait',
'Face',
'Photography',
'Person',
'Face',
'Photography',
'Person',
'High Rise',
'City',
'Urban',
'Building',
'Road']Let’s say that we want to get the unique labels of the parent list. We can simply use the set function.
set(parent_lst)
{'Building',
'Car',
'City',
'Face',
'High Rise',
'Person',
'Photography',
'Portrait',
'Road',
'Street',
'Transportation',
'Urban',
'Vehicle'}Notice that by taking the labels and the parent label of the images you can build a Machine Learning Model to measure the performance of the images where your features will be the labels. You can then apply tf-idf on them and generally treat them as an NLP task.
Images on a Local File System
Below we represent the same example as above, but this time by reading the image from the local file system.
import boto3
client=boto3.client('rekognition')
# My photo
photo = 'skateboard_thumb.jpg'
with open(photo, 'rb') as image:
response = client.detect_labels(Image={'Bytes': image.read()}, MaxLabels=50)
# get a list of labels
label_lst = []
for label in response['Labels']:
label_lst.append(label['Name'])
# get a list of parents
parent_lst = []
for label in response['Labels']:
for parents in label['Parents']:
parent_lst.append(parents['Name'])
References
- https://docs.aws.amazon.com/rekognition/latest/dg/labels-detect-labels-image.html
- https://docs.aws.amazon.com/rekognition/latest/dg/images-bytes.html?fbclid=IwAR0Q-7kJCb-NYyDWIDoXTj6l0hXnkzF6tJVhW2BZj7m5EPfE1ny3-Ya7OT4
