Sentiment Analysis with AWS Comprehend in Python

In the past, we have explained to you how to run a Sentiment Analysis with VADER and Sentiment Analysis with Naive Bayes. Also, we have provided you with an example of AWS Comprehend of how to redact PII data. In this tutorial, we will show you how to run a sentiment analysis using the AWS Comprehend with the AWS SDK for Python.

Note that I have several AWS accounts and profiles. We have shown how to choose an AWS profile in Boto3. Let’s start coding by trying the following input:

his place is extremely expensive and the owner is very rude 🙁

import boto3

session = boto3.session.Session(profile_name='sandbox')
client = session.client('comprehend')
 

mytxt = "This place is extremely expensive and the owner is very rude :("
 
response = client.detect_sentiment(
    Text= mytxt,
    LanguageCode='en'
)
 

# get the response
 
response

Output

{'Sentiment': 'NEGATIVE',
 'SentimentScore': {'Positive': 0.0002594548568595201,
  'Negative': 0.9980950951576233,
  'Neutral': 0.001384085277095437,
  'Mixed': 0.0002613658725749701},
 'ResponseMetadata': {'RequestId': 'cd4252b7-4281-48fc-b477-b48f0d1ad366',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'x-amzn-requestid': 'cd4252b7-4281-48fc-b477-b48f0d1ad366',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '165',
   'date': 'Wed, 06 Apr 2022 10:20:47 GMT'},
  'RetryAttempts': 0}}

As we can see, the input was definitely a negative one, and this is aligned with what we got from the AWS Comprehend. Note that the output is JSON format and we can easily extract the keys. Let’s say that we just want to get the sentiment.

response['Sentiment']

Output

'NEGATIVE'

Working with Pandas

Almost all Data Scientists love Pandas. Assume that we have the reviews in a Pandas Data Frame and we want to add an extra column with the Sentiment Score:

# create a dummy data frame
import pandas as pd

df = pd.DataFrame({'reviews':["I'm grateful. Thank you!!!", "I'm really disappointed"]})

df

# create a function
def get_sentiment(mytxt):
    
    response = client.detect_sentiment(
    Text= mytxt,
    LanguageCode='en')
    
    return response['Sentiment']

Now we are ready to get the sentiment score using a lambda function.

df['sentiment'] = df['reviews'].apply(lambda x: get_sentiment(str(x)))
df

Attention!

The AWS Comprehend is not free of charge, and you are getting charged for every API call. Thus, if you want to go with a free solution, we recommend working with VADER