How to use Alexa with Built.io Flow

Amazon Alexa is an intelligent voice assistant that powers Amazon Echo – voice-enabled smart speaker developed by Amazon.com. It lets you give voice requests and get a custom response.

When this device is integrated with Built.io Flow, you can create amazing workflows to automate many routine tasks by just giving voice commands.

Let’s create a sample workflow to understand how Amazon Alexa works with Built.io Flow.

We will set this up in such a way that whenever you request Alexa for unread messages (by saying 'Alexa ask mailer to get count of unread messages from XYZ' or similar sentences), this workflow will get triggered. The workflow will first retrieve the unread messages from a specific person in Gmail and will read them out to the user via Amazon Alexa.

However, before creating the workflow, you need to first set up the custom skill in Amazon Alexa for the request to be performed:

Creating a Custom Skill

Custom skill determines the way you can interact with Alexa and perform any particular action using 'Custom Interaction Model'. You can learn more about custom skills here.

To set up a custom skill, you need to have an ‘Amazon Developer’ account. log-in to your account, and click the ‘Alexa’ tab on the home page. You will be redirected to a new page where you’ll need to select the service that you wish to use for voice interaction with Alexa. Click on the ‘Get Started’ button given under ‘Alexa Skills Kit’.

You will be redirected to a new page. Click on ‘Add a New Skill’ button.   

You will now see a configuration window for creating a custom skill. The configuration window has six sections in the navigation panel. You need to fill out each one in order to create a custom skill. Details of these sections are listed below:   

1. Skill Information: Enter the basic information about your custom skill as given below:

   - Skill Type: Select ‘Custom Interaction Model’ radio button, as a skill type.

   - Name: Enter the name of the skill that will be displayed to your users in Alexa app.

   - Invocation Name: Define the invocation name by which Alexa will identify the skill. The invocation name must be included in the request made by user. In our example, the invocation name for the skill that fetches the response from 'Gmail' service is called 'Mailer'. 

Once you have entered the details, ‘Save’ them and click on ‘Next’.

2. Interaction Model: Define the intent schema, slots (if any), and sample utterances in this section as given below:

   - Intent: Define the intent (request) your skill will handle. A skill can include multiple intents. Given below is the intent schema for our example:

  - Sample Utterances: Define the words or sentences the user may use to invoke the intents/requests. It is important to note that all the sample utterances should include the name of the intent that you wish to evoke. For our example the sample utterances are:

Once you have entered the details, ‘Save’ them and click on ‘Next’.

3. Configuration: Provide the webhook URL where you will post the response for Alexa.

   - Endpoint: Since we are using HTTP post method for posting data on Webhook URL, select ‘HTTPS’ radio button and enter the ‘Webhook URL’ of your workflow (Learn more about how to retrieve the webhook URL here) in the text box given below.

 It is important to note that once you have added 'Return Data on Sync Webhook' action to your workflow, '?sync=true' is automatically appended to your webhook URL. (The process of creating the workflow is explained in the later section). This syncs the workflow with webhook URL and sends the workflow output to it when the webhook URL is hit. So when you are entering the 'Webhook URL' of your workflow in the 'Endpoint' field, please make sure that '?sync=true' is appended to it. 

   - Account Linking: Specify if you wish to allow users to create an account or link to an existing account with you. 

4. SSL Certificate: Select the method you will be using for the web service.

Once you have selected the web service method, ‘Save’ it and click on ‘Next’. You will be redirected to the ‘Test’ section where you can test your custom skill. Once the testing is done, you can fill out the details for ‘Publishing Information’ and ‘Privacy and Compliance’ sections and submit the skill for certification. 

Once it passes Amazon’s testing and certification process, it will be available under ‘Skills > Your Skills’ page of the Alexa app. You can then enable it and start using this skill. 

The user request to Alexa should be structured as follows:

User: "Alexa ask mailer to get count of unread messages from boss".

Now we will create a workflow that will handle this request.

Creating a Workflow

We will create the workflow shown below to handle the request made by the user.

We will create this workflow in such a way that it handles two cases:

1. There are ‘X’ unread messages from a specific person and Alexa will return these messages in response.

2. There are no unread messages from a specific person. Alexa will accordingly provide a suitable message in response.

Let’s first quickly look at the actions that we will be using in this workflow:

Actions used in Case 1

1. Gmail_Search Mails: Retrieve all unread messages from a specific person from your Gmail account.

2. Condition_1: Check if there are any unread messages from a specific person. If this condition is met, the workflow will proceed to the next action. 

3. Group: Run all the actions added inside ‘Group’ in loop for each element of the input array. 

4. Gmail_Get Mail Details: Retrieve all the details of a current message.

5. Node.js Code: Write a code to extract and store the ‘Subject’ of the current message.

6. Node.js Code: Write a code to format a response for Alexa depending upon the output of the ‘Group’ action. 

7. Amazon Alexa_Response for Alexa: Format the output response in such a way that Alexa understands and requires to respond.

8. Return Data on Sync Webhook: Fetch the response from ‘Amazon Alexa’ action and send it to the webhook, which in turn sends it to Alexa as a response for the request made by the user. 

Actions used in Case 2

1. Gmail_Search Mails: Retrieve all unread messages from a specific person from your Gmail account.

2. Condition_2: Check if there are no unread messages from a specific person. If this condition is met, the workflow will proceed to the next action. 

3. Node.js Code: Write a code to format a response for Alexa stating there are no unread messages from a specific user. 

4. Amazon Alexa_Response for Alexa: Format the output response in such a way that Alexa understands and requires to respond.

5. Return Data on Sync Webhook: Fetch the response from ‘Amazon Alexa’ action and send it to the webhook, which in turn sends it to Alexa as a response for the request made by the user. 

6. Enable Webhook: Enable the webhook URL to which you will be posting the response for Alexa.

Let us now understand how to go about creating this action. 

Step 1: Add ‘Gmail_Search Mails’ action

This action will retrieve unread messages from a specific person from your Gmail account. To do this, add 'Gmail_Search Mails' action to your canvas and configure it as given below:

Field data

  - Gmail Access Token: Select your Gmail access token from the drop-down list. If you have not created an access token yet, click on 'Add new' to create one.

  - Max Results: Specify the number of results you wish to include in the response.

  - Filter Keyword: Enter the email address of the user whose unread messages you wish to retrieve in this format: 'from:someone@example.com is:unread'.

  - Next Page Token Enter the ‘Next Page Token’ value to retrieve the records of a specific page. You can retrieve the ‘Next Page Token’ value from the response of ‘Search Mails’ action. 

Once you have entered all the details, click ‘Next’, and test the action by clicking on ‘Test’ button.

This serves two purpose: First, whether the action has been configured properly, and second, it returns one of the unread mails as an output. This action has two possible outcomes:

   Case 1: There are 'x' unread messages from a specific person and Alexa will return these messages in response.

   Case 2: There are no unread messages from a specific person and Alexa will return a response stating the same. 

We will now create a workflow that covers these possibilities. 

Steps for Case 1 

Step 2: Add ‘Loop’ action

The ‘loop’ action will run all the actions added inside the loop block for each element of the input array. To do this, add a 'loop' action to the canvas and connect it to the 'Gmail_Search Mails' action. 

Now configure it as shown below:

Field data

  - Select Loop Type: Select the 'For Each' loop type from the drop-down list.

  - Array OR Object: Add the output of 'Gmail_Search Mails' action given on the right-hand side of the configuration window, as an input for this field. You can see the actual values of the action against the keys, and also the data type of values.

Once you have entered all the details, click 'Next', and test the loop action by clicking on the 'Test' button. Similarly, it will check whether the loop action has been configured properly. Click on 'Done', to go back to your canvas.

Step 3: Set a condition

Set a condition that checks if you have any unread messages from specific person. If this condition is met, the workflow will proceed to the next action. To do this, click on the line connecting 'Gmail_Search Mails' and 'Loop' action. You will see three icons; 'Settings', 'Delete', and 'Detach'. Click on 'Settings' icon. A new 'Condition' window will appear. Click on '+CONDITION' link and set the condition as given below:

     - Input : Add the output of the 'Gmail_Search Mails' action given on the right-and side of the configuration window, as an input for this field.

    - Condition: Select '(Text) Does Not Match' from the drop-down list.

    - Expected value: Enter 'undefined' as an input for this field. 

Here, we have set a condition that the input (output of previous action) should not be undefined. This means that the workflow will proceed to the next action if the output is not undefined. 

Step 4: Add ‘Gmail_Get Mail Details’ and ‘Node.js Code’ actions inside ‘Loop’

The ‘Gmail_Get Mail Details’ action will retrieve all details of the current message from the input array, and ‘Node.js Code’ action will extract and store the ‘subject’ from the details fetched by the ‘Gmail_Get Mail Details’ action. 

The ‘Loop’ block will run both the actions in loop for each item of the input array. To do this, add and connect 'Gmail_Get Mail Details' and 'Developer Tools_Node.js Code' actions inside Loop action as shown below:

Now, configure the 'Gmail_Get Mail Details' action as given below. This action will retrieve the details of a current message from the input array. 

Field data

  - Authorize Gmail: Select your Gmail authorization from the drop-down list.

  - Message ID: Add the ID of the currentItem '{{$a1.currentItem.id}}' key listed under the 'Loop' action output parameters as an input for this field. This key returns only the message ID of the input array. You can see the actual values of the loop action against the keys, and also data type of values.

Once you have entered all the details, click on 'Next', and 'Test' your action. This will serve two purpose: First, it will check whether the action has been configured properly, and second, it will return the latest unread mail details. 

Click on 'Done' to return back to the canvas.

Next, write a code using 'Node.js Code' action that extracts and stores the subject of the current message from the details retrieved by the 'Gmail_Get Mail Details' action.

Code

   var subject;

   $ga10.payload.headers.forEach(function(header) {

       if(header.name === "Subject")

           subject = header.value;

   });

   if($local.subjects){

       $local.subjects.push(subject);

   }else{

       $local.subjects = [];

       $local.subjects.push(subject);

   }

   $export(null, subject);

Workflow will continue running these two actions in loop for each element of the input array. Once it has iterated over the last element, the workflow will proceed for the next action. 

Step 5: Add ‘Node.js Code’ action 

Write a code using 'Node.js' that formats a response for Alexa depending upon the output of the loop action. 

Code

   var speechText = "No unread messages from your boss were found";

  if($local.subjects && $local.subjects.length>0) {

       speechText = "There are : " + $local.subjects.length + " unread messages from our boss :"

       for(i = 0;i<$local.subjects.length;i++) {

           speechText+= " : subject " + (i+1) + " : is " +$local.subjects[i];

       }

   }

    $export(null, speechText);

Step 6: Add ‘Amazon Alexa_Response for Alexa’ action

This action will format the output from the previous action in a way that Alexa understands and requires to respond. To do this, add this action to canvas and configure it as shown below. 

Field data

  - Speech Type: Select 'PlainText' from the drop-down list as the speech type for your response 

  - Enter Response Speech: Add the output of the 'Node.js Code' action given on the right-hand side of the configuration window as an input for this field.

Now, click on 'Next', and test the action by clicking on the 'Test' button. Now that you are done with testing the action, click on 'Done' to return back to the canvas.

Step 7: Add ‘Return Data on Sync Webhook’ action

This action fetches the response from 'Amazon Alexa' action and sends it to a webhook, which in turn sends it to Alexa, as a response for the request made by the user. Please note that every 'Response for Alexa' action is followed by 'Return Data on Sync Webhook' action . To do this, add this action to canvas and configure it as given below:

Field data

 - Response: Add the response from 'Amazon Alexa' action given on the right hand side of the configuration window as an input for this field.

 - Status Code: Enter '200' as the status code which is a standard response for successful HTTP calls.

 - Content Type: Select 'application/json' content type from the drop down list. 

Once you have entered all the details, click 'Next', and test the action by clicking on the 'Test' button. Testing serves two purpose: First, whether the action has been configured properly, and second, it returns the sample response for Alexa. 

Click 'Done', and connect this action to the 'Stop' icon.

This workflow will fetch all the unread messages from a specific person from your Gmail account. We will now create a workflow for the second case where there are no unread messages from a specific person. 

Steps for Case 2 

 Step 8: Add ‘Node.js Code’ action

Write a code using 'Node.js' block that forms a response for Alexa stating there are no unread messages from a specific person.

Code

Step 9: Set a condition

Set a condition that checks if there are no unread messages from a specific person. If this condition is met, the workflow will proceed to the next action . To do this, hover the mouse over the line connecting 'Gmail_Search Mails' and 'Node.js Code' action and click on it. You will see four icons; 'Settings', 'Delete', 'Detach', and 'Add Note'. Click on 'Settings' icon. A new 'Condition' window will appear. Click on '+CONDITION' link and set the condition as given below:

Field data

     - Input: Add the output key 'messages' under the 'Gmail_Search Mails' action given on the right-hand side of the configuration window, as an input for this field.

    - Condition: Select '(Text) Matches' from the drop-down list. This condition will check if there are no unread messages from a specific person. If this condition is met, the workflow will proceed to the next action . 

    - Expected Value: Enter 'undefined' as an input value for this field. 

Click 'Done' to return back to canvas.

Step 10: Add ‘Amazon Alexa_Response for Alexa’ action

This action will format the output from the previous action in a way that Alexa understands and requires to respond. To do this, add this action to canvas and configure it as shown below:

Field data

  - Speech Type: Select 'Plaintext' from the drop-down list as the speech type of the output response. 

  - Enter Response Speech: Add the output of the 'Node.js Code' action given on the right-hand side of the configuration window as an input for this field.

Once you have entered all the details, click 'Next', and test the action by clicking on the 'Test' button. This serves two purpose: First, it checks whether the action has been configured properly, and second it will return a sample response.

Click 'Done' to return back to the canvas.

Step 11: Add ‘Return Data on Sync Webhook’ action

This action fetches the response from 'Amazon Alexa' action and sends it to a webhook, which, in turn, sends it to Alexa as a response for the request made by the user. Please note that every 'Response for Alexa' action is followed by 'Return Data on Sync Webhook' action. To do this, add this action to canvas and configure it as given below:

Field data

  - Response: Add the response from 'Amazon Alexa' action given on the right-and side of the configuration window as an input for this field.

  - Status Code: Enter '200' as the status code which is a standard response for successful HTTP calls.

Once you have entered all the details, click 'Next', and test the action by clicking on the 'Test' button. This serves two purpose: First, it checks whether the action is configured properly, and second it returns the sample response for Alexa.

Click 'Done' to return back to the canvas, and connect this action to the 'Stop' icon.

Now, when a user ask Alexa to check unread messages from a specific person, the workflow will first access his Gmail account to see if there are any unread messages from a specific person and will return the corresponding response through Alexa.