10.1: Analyze sentiment with automations and AWS Comprehend

[jstanden]

Introduction

In Cerb you can create workflows that depend on the sentiment of an inbound message. For instance, you can ask positive messages to provide a testimonial, while giving negative messages some extra personal attention.

If you send surveys that rate customer satisfaction on a scale of 0-10, it's pretty easy to determine their sentiment. However, sometimes all you have is some text like an email reply or a comment. In this case, you can use a machine learning model to automatically label the sentiment of the text.

Cerb's classifiers could be used to build a sentiment analysis model; however, that requires a decent amount of labeled training data to start making good predictions.

Instead, in this guide we'll be integrating with the Comprehend service from Amazon Web Services. This has built-in models for keyphrase extraction, sentiment analysis, syntax analysis, entity recognition, and language detection. You can get reasonably accurate real-time predictions from a pre-trained model.

We'll be using this service through its HTTP API. You can use this same approach for any other machine learning service (Google Cloud Natural Language API, Hugging Face, etc).

Create a connected service in Cerb for AWS

Log into Cerb as an administrator.

If you haven't already, create a connected service/account for Amazon Web Services with these instructions:
https://cerb.ai/guides/integrations/aws/

Update the policy on your IAM user to allow the comprehend:DetectSentiment action.

Import the sentiment interaction

We've created a package to demonstrate this integration.

Navigate to Setup -> Packages -> Import.

Paste the following package:

{
  "package": {
    "configure": {
      "prompts": [
        {
          "type": "chooser",
          "label": "AWS Account",
          "key": "aws_account_id",
          "params": {
            "context": "cerberusweb.contexts.connected_account",
            "single": true,
            "query": "service:(name:amazon)"
          }
        },
        {
          "type": "text",
          "label": "AWS Region",
          "key": "aws_region",
          "params": {
            "default": "us-west-2"
          }
        }
      ]
    }
  },
  "records": [
    {
      "uid": "automation_sentiment",
      "_context": "automation",
      "name": "wgm.example.comprehend.sentiment",
      "description": "Detect the sentiment of an email message",
      "extension_id": "cerb.trigger.interaction.worker",
      "script": "inputs:\n  record/message:\n    record_type: message\n    required@bool: yes\n\nstart:\n  set:\n    request_body:\n      LanguageCode: en\n      Text: {{inputs.message.content|strip_lines(prefixes='>')|truncate(5000)}}\n  \n  http.request:\n    # Make a request to the AWS Comprehend API\n    output: http_response\n    inputs:\n      method: POST\n      url: https://comprehend.{{{aws_region}}}.amazonaws.com/\n      authentication: cerb:connected_account:{{{aws_account_id}}}\n      headers:\n        Content-Type: application/x-amz-json-1.1\n        X-AMZ-Target: Comprehend_20171127.DetectSentiment\n        X-AMZ-Date: {{'now'|date('Ymd\\\\THis\\\\Z', 'UTC')}}\n      body: {{request_body|json_encode}}\n    on_success:\n      await:\n        form:\n          title: Sentiment Analysis\n          elements:\n            say:\n              content@text:\n                ```\n                {{http_response.body|json_pretty}}\n                ```\n  ",
      "policy_kata": "commands:\n  http.request:\n    deny@bool:\n      {{\n        inputs.url != 'https://comprehend.{{{aws_region}}}.amazonaws.com/'\n        or inputs.method != 'POST'\n        or inputs.authentication != 'cerb:connected_account:{{{aws_account_id}}}'\n      }}\n    allow@bool: yes"
    }
  ],
  "toolbars": [
    {
      "toolbar": "mail.read",
      "kata": "interaction/sentiment:\n  label: Sentiment\n  uri: cerb:automation:wgm.example.comprehend.sentiment\n  icon: scale-classic\n  inputs:\n    message@key: message_id"
    }
  ]
}

Click the Import button.

Select your AWS connected account and region, then click Import again.

You should see the following results:

Create test data

Now let's create four test tickets for each sentiment outcome: positive, negative, mixed, and neutral.

This guide assumes you're using a test environment. In production, you can use live tickets and skip this step.

Navigate to Setup -> Mail -> Incoming -> Import and paste the following messages.

Positive:

From: [email protected]
To: [email protected]
Subject: I like Cerb

As a newly remote team, we started managing all of our projects and email with Cerb. We're getting more done than before. The new automations also look incredible and we're eager to start using them.

Negative:

From: [email protected]
To: [email protected]
Subject: I don't like Cerb

Cerb is the kind of special mess you get when programmers try to be designers. Have you ever user tested anything in your entire life? This is easily the worst app I have ever tried to use.

Mixed:

From: [email protected]
To: [email protected]
Subject: Cerb is not for me

I can tell that Cerb is powerful, but it's way too complicated for my team. I feel like we'd have to be expert programmers to use any of the functionality. I'm going back to Gmail.

Neutral:

From: [email protected]
To: [email protected]
Subject: I am merely describing Cerb

Cerb is a web-based work management and automation platform for remote teams.

Analyzing sentiment

Navigate to the profile of any of the four new test tickets above (or your own data) from Search -> Tickets.

You'll see a new Sentiment button in the mail.read toolbar that was added by the package above.

Click the button to view the sentiment analysis:

Next steps

This is a simple interaction to review the results of sentiment analysis for any message without taking any actions.

Instead, you could automatically analyze the sentiment of all incoming messages.

Conclusion

You now understand how to integrate sentiment analysis from AWS Comprehend into Cerb using automations.