How to set up Entity Recognition in Flow Builder

In this guide, you will learn how to use the Recognize Entities step in Flow Builder to automatically extract entities from customer messages and use them to improve your flows.

Requirements

• A MessageBird account

• WhatsApp for Business account or any other channel from MessageBird's offering

What’s Entity Recognition?

Named Entity Recognition (NER) is the task of identifying and categorizing entities (key information) in text. An entity is something that often refers to the same thing. MessageBird’s Recognize Entities step uses machine learning models to detect these entities. For example, it might recognize the date in “I want to book a ticket for next Friday” or the email-address in “You can reach me on [e-mail]”.

Use the demo in the Flow Builder step to check out more entities. In combination with our Recognize Intent step, you can create some powerful applications built upon natural communication with your users.

Supported entity types

The Recognize Entities step supports the following entity types:

Entity Type	Flowbuilder entity	Example
Amount of Money	amount-of-money	€42
Credit Card Number	credit-card	4111-1111-1111-1111
Distance	distance	10 kilometre
Duration	duration	3 mins
Email	email	[e-mail]
Numeral	numeral	88
Ordinal	ordinal	33rd
Phone Number	phone-number	[phone]
Temperature	temperature	38 degrees
Time	time	Today at 9 AM
URL	url	https://www.messagebird.com
Volume	volume	3 Gallons
Interval	interval	From 9 to 5
Person	person	Robert Vis
Geopolitical entity (Cities, countries/regions, states)	geopolitical	San Francisco
Quantity	quantity	3 cups

How does it work?

To recognize entities our Recognize Entities step uses an advanced (probabilistic) language parser in combination with Machine Learning to extract and find entities. It does so by learning from lots of examples of how an entity is used and in which contexts. It then uses a rule-based parser to turn the found entities into an expanded version (more explained on that below).

Setting up an Entity Recognition flow

In this first part of the quickstart, we’ll set up a flow for the fictional company CoffeeBird. The flow uses the Recognize Entities step to allow users to place a coffee order in an autonomous way.

Step by step

1. Log in to your MessageBird Dashboard and go to Flow Builder.

2. Select a flow (or create a new one). For this example, we’re going to create a new custom flow with an Omni-channel trigger.

3. Create a dataset and intent recognition step according to the Intent Recognition - Quickstart. The dataset should have two intents: “order” and “book”.

4. Done? Awesome! Set up branching between the intents—we’ll only continue with the “Order” intent here.

5. Reply (using the Reply to channel message step) with: “How many coffees would you like to order?”

6. Add a Wait for a response step and call the output variable of the response coffee_amount_message.

7. Create a Recognize Entities step. Set the ‘Content to recognize’ to coffee_amount_message. Then, configure it to recognize numerals by clicking Add entity and setting the entity to numeral (we can use the variable name coffee_amount here). And save both the entity and the step.

8. The user should now be able to answer with “I’d like two!” and “2” should end up in coffee_amount. You can verify your flow by replying to the channel message with “You ordered coffee_amount.normalized (coffee_amount.text) coffee” with the Test feature next to the Save draft and Publish buttons.

This step shows the attribute syntax (coffee_amount.normalized instead of just coffee_amount) that is sometimes used in Flow Builder, you might have seen it before when working with the contact variable. When recognizing entities, you probably want to do something with them. For instance, use a customer number as a tag in Inbox, enrich your contacts using a recognized email, hand a custom product number over to your CRM, or use the desired delivery date to update your order fulfillment software. However, most systems won’t understand the date of the text “Next week friday” but will require more specific formatting. That’s what coffee_amount.normalized is providing here.

Later in this quickstart, we will tell you more about the expansion of these variables. To get access to coffee_amount.normalized start by typing {{ and select the coffee_amount variable and the normalized attribute.

9. Reply to the channel message with: “When would you like to come to pick your coffee up??”

10. Add a Wait for a response step (pickup_time).

11. Create a Recognize Entities step. Configure it to recognize time by going through the same process as in step 7 while choosing the right entity and variable names. This should recognize time in a message like: “At 3:00 pm” or even “In 2 hours”.

12. Reply to the message with: “We’ll have it ready at time.text (time.normalized)”

13. You can now use the information you’ve collected in the conversation to start your order using a webhook or set it up as an Inbox ticket!

14. Publish your flow in the top-right corner of your screen, and you're good to go!

Good practices

Check whether the Recognize Entities step found entities before continuing. A Jump to step might help here to go back up the flow and have the user send another message. See below for an example.

Expansion

In step 8 of Setting up an Entity Recognition Flow, we showed you how the entity recognition attributes work. As explained, normal computer systems will not know how to work with text such as “Next friday at 14:00” but do understand the ISO-version of the date “2020-09-18T14:00:00.000+00:00”. That’s why our Recognize Entities step does this automatically, but still gives you access to the text version as well. You can access them for a variable called time by using {{time.text}} and {{time.normalized}}.

Not all entities are able to be expanded. The ones where it is possible are: time, interval, duration, and numeral. The table below shows how they get expanded. Other entity types do not get expanded and the .normalized version will be filled with the same value as the .text version

Entity Type	Example	variable.text	variable.normalized	Normalized format
time	I’ll be there at 14:00	14:00	2020-09-09T14:00:00.000+00:00	ISO-date Year-Month-DayTHour:Minute:Second.Milliseconds+Timezone
interval	It will be delivered between 3pm and 16:00	between 3pm and 16:00	2020-09-09T15:00:00.000+00:00,2020-09-09T16:01:00.000+00:00	ISO-date-from,ISO-date-to
duration	It will take 3 hours	3 hours	3 hour	Value unit Unit: Hour, minute, second, day
numeral	I’ll take three	three	3	Value

Adding custom coffees

In this second part of the quickstart, we’ll set up custom entities to have CoffeeBird sell not only coffee but also more exotic variants like a Cappuccino and a Flat White. We’ll extend the previous flow to ask the user what kind of coffee they want to order.

First, we’ll set up the custom entities (these are the custom/specific words that we will be looking for within incoming messages). There are two types of Custom Entities:

• Keywords: This is the type that we’re going to be using in our example. It has a list of keywords that it extracts, e.g. our coffee types. It has the ability to recognize them even with some spelling errors. These keywords also are expanded in the same way that our time entity type is. Every keyword can have variants that will be expanded to the same keyword. For example in the context of CoffeeBird “Black” can be expanded to “Coffee”. Variants are added by using the plus sign to the right of a keyword.

• Regular Expressions: This type is all about extracting entities that follow a specific format, e.g. product number like “MB:NL:102”. To learn more about regular expressions and test your regular expressions take a look here and select the “Python” flavor. The exact syntax used by our regular expressions can be found here.

You might use a Regular Expression to match a Dutch postal code by using this regular expression: [1-9]{1}[0-9]{3}\s?[a-zA-Z]{2}

Step by step

Step 1: Set up your custom entities

1. Log in to your MessageBird Dashboard and go to Knowledge Bases

2. In the list of datasets, there is an item called Custom Entities. Here we can set up our brand new coffees!

3. Add a Custom Entity and name the new group of custom entities “Coffee types”.

4. Choose Keywords on the left and add the following keywords: “Coffee”, “Cappuccino”, “Flat White”, “Espresso” and “Americano”. To extend the functionality of our entity recognizer we will add a variant for our “Coffee” keyword. Use the plus sign to the right of our “Coffee” keyword to add a variant. Fill the variant for instance with “Black”.

5. Save your Custom Entities.

Done? Awesome! Let’s extend the flow to start using them!

Step 2: Extend your flow

1. In the “Order” branch add an additional Reply to channel message with the text “What kind of coffee would you like to order? Our menu consists of Coffee, Cappuccino, Flat White, Espresso and Americano”

2. Add a Wait for a response step.

3. Create a Recognize Entities step below it. Configure it to recognize the custom entity “Coffee types”. The step will pick up the coffee entities we just defined even if your customer misspells them slightly!

4. Change the Reply to channel message step below to include the Coffee type by using the text: “How many coffee_type.normalized would you like to order?”. Notice in the image below that the misspelling “Capucino” still gets picked up as the correct coffee type!

5. Publish your flow to get the coffee orders to come in!

And that’s it! 🎉 Now you know how to set up Entity Recognition in Flow Builder.