Connector Builder

Built.io Flow is an integration platform as a service (iPaaS) that lets you connect web apps, on-premise systems, and devices to create integrations and automate tasks.

You can create custom web connectors for Built.io Flow using the Connector Builder.

What is a Connector Builder App?

The Connector Builder app is a custom Node.js application that you build using a web application’s APIs. So, for example, if you have a private API or an API that is not in Flow yet, you can create custom Flow actions and triggers for those APIs. Once you have created these custom actions and triggers, you can use them like any other action in Built.io Flow.

How the Connector Builder app works?

Once you deploy the app on Built.io Flow, it is validated by Built.io Flow. If the app contains any errors, you will be displayed the relevant error message and error details on the console. Once the app is validated successfully, it will be  made available to you locally. You can then share this app with specific users or publish it to Built.io Flow to make it available to all the users of Built.io Flow. Below is the list of basic terms you should be familiar with before you start creating your first Connector Builder app

1. Trigger: It reads data from the specified API and can be used to execute a workflow execution.
2. Action: It sends data to the specified API to perform CRUD operations.
3. Authentication: It specifies what credentials are required from the user.
4. Lookup: It searches for specific records in your system or account. 

Connector Builder vs Custom Action

Both the Connector Builder and the existing ‘Custom Action’ feature of Built.io Flow serve the same purpose; they let you create custom connectors. The major difference is that the Connector Builder works more like a programming project where you have to create your application from scratch whereas custom action offers you a ready-to-use interface where you just have to enter the core logic for your application.

However, the Custom Action features lets you create only actions, while the Connector Builder lets you create actions and triggers along with the custom authentications. You can choose any one of these options that fits your requirements and start creating your apps!

Requirements

The app can be developed using any Node version between 4.3.2 and 6.10.X.

If you have installed any other Node version (for example, v8.9.4.) on your system, you can choose from one the following options:

1. Switch to one of the Node versions between 4.3.2 and 6.10.x using tools such as node-windows, nvm, n, and run the app locally.

 - For Windows, use:

        -  nvm-windows

        -  nodist

 - For nvm on Mac (via homebrew), use:

        - brew install nvm

        - nvm install v6.10.2

OR

2. Build app using your existing Node version (for example, v8.9.4.) and then transpile it using Babel or similar transpiler.

Installing ‘@builtioflow/connector-builder’ Package

In order to get started with the Connector Builder, you will first need to install it on your system. To do so, run the following in the command prompt:

npm install @builtioflow/connector-builder -g  

It will automatically install the package in your current directory. Once this is done, you can start building your app!

Quick Setup Guide

If you are familiar with how an iPaaS Connector Builders works, you can use the quick start guide below to start building your connectors. (For a detailed explanation of these steps, jump to ‘Creating Local App’ section)

Step 1. Install the Connector Builder globally

npm install @builtioflow/connector-builder -g

Step 2. Login to Built.io Flow

flow login

At this point your Connector Builder is installed and ready to go!

Step 3. Create a new app with minimum required files

flow init

You will be prompted to provide a name for the app. 

Step 4. Move into newly created directory

cd <app_name>

Step 5. Install all libraries needed for your Connector Builder app

npm install

You should now have a local working app. You can now start adding codes for your actions/triggers/lookups and authentication.

Step 6.  Add action,trigger, and lookup

To add new action in your application, use

flow create action <action_name>

When you execute this command, a new ‘action’ folder will be created in your app’s directory with a ‘<action_name>.js’ file in it. You can then add your action code in that file’s execute function and update the input and output schema based on your requirements.

To add new lookup in your application, use

flow create lookup <lookup_name>

When you execute this command, a new ‘lookup’ folder will be created in your app’s directory with a ‘<lookup_name>.js’ file in it. You can then add your lookup code in that file.

To add new trigger in you application, use

flow create trigger <trigger_name>

Note: The action/trigger/lookup name should consist of only alphanumeric characters and underscores.

Step 7.   Create authentication for actions/triggers/lookup. 

flow auth

Step 8. Once all codes are added, test the app. 

This is an optional step but we recommend that you perform it. The testing will be done against the mock data you have provided in your code.

flow test

Step 9. Deploy the app to Built.io Flow.

flow deploy

Once you have deployed the app, refresh the browser window of Built.io Flow UI. The deployed app will be added in the ‘Actions’ panel under the ‘Custom’ tab and will be available for you to use locally. You can then download,share/unshare, or publish this app.

Note: Refer to the ‘Help’ section to get more details about the Connector Builder commands)

In the next section, we will understand how to create a Connector Builder app in detail.

Creating a Local App

Once you have installed the ‘@builtioflow/connector-builder’ package, you can install the Connector Builder and set up your authentication to create a working app. It will be private and visible in your ‘Actions’ panel under the ‘Custom’ tab. Before we go through the steps to create a local app, let’s understand the important blocks that make the app.

Triggers

Triggers read data from the external app and execute the related workflow in Built.io Flow. Learn more about triggers and how they work.

The basic structure for creating a trigger using the Connector Builder is given below. You can add your custom code in this structure to create your own triggers. Read more on how to create triggers with Connector Builder.

module.exports =
{
 name: "sample_trigger",

 input:
 {
   type: 'object',
   title: 'Sample Trigger',
   description: "Short description",
   properties:
   {
     event:
     {
       type: 'string',
       enum: ['sample_trigger']
     }
   }
 },

 output:
 {
   "sample_trigger":
   {
     type: 'object',
     properties: {}      
   }
 },

 mock_data: {}, // add sample output data that will be used for testing

 execute: function(input, options, output)
 {
   // will be called every 5 minutes
   // your code here
 },

 activate: function(input, options, output)
 {
   // This function will be called whenever user activates the trigger/workflow. 
  // This function is useful in scenarios where you save cursor to figure out newly added/changed data in the
  // 3rd party services. You can update your cursor so that trigger won't fetch data of the period 
  // during which the trigger was deactivated.
 },

 validate: function(input, options, output)
 {
   // This function will be called when the trigger is created. 
   // This functions validates all inputs provided by the user and sends an error if any of the details (including auth) are incorrect. In case of an error, trigger server won't create the trigger.
 }

}

Actions

Actions define the tasks to be performed. The basic structure for creating an action is given below. You can add your custom code in this structure to build your own actions. Read more on how to create actions with Connector Builder.

module.exports =
{
 label: "sample_action",
 description: "",

 input:
 {
   title: 'sample_action',
   type: 'object',
   properties: {}
 },

 output:
 {
   title: 'output',
   type: 'object',
   properties: {}
 },

 execute: function(input, output)
 {
   // your code here
 }

}

Authentications

Most of the actions/triggers need some sort of authentication from users to function. Built.io Flow provides some methods to help add authentication for your actions and triggers. These methods are listed below:

1.Basic

If your action/trigger only needs two pieces of information from the user, i.e., ‘Email ID’ and ‘Password’, you can use the basic authentication method. Below is the structure that should be used for basic authentication. You can add your custom code in this structure to create your own basic authentication mechanism. View the template for basic authentication or read more on how to create basic authentication with Connector Builder.

module.exports = 
{
  label : 'Connect to Demo',

  mock_input: {
    "username" : "myuser",
    "password" : "mypassword"
},

  validate : function (input, output)
  {
    // auth data will be available in input.auth
    // var username = input.auth.username
    // var password = input.auth.password
    // make call to third party api to validate user credentials
    // and return callback output(null, true) in case of successful validation
    // or return callback output(‘error message’) in case of unsuccessful validation
  }

}

2. OAuth

OAuth is a token-based authentication method that allows the user’s account information to be used by any external service, and subsequently provide secure access, without exposing the password. For example, many websites offer login multiple methods. One of the login methods you commonly see is ‘Login via Gmail’ or ‘Login via Facebook’. This is an example of OAuth. . Given below is the basic structure for the Oauth authentication.

Built.io provides built-in OAuth authentication modules for a number of services. You can find the service list here. You can either use these built-in modules or can create custom OAuth authentication modules as per your requirements.

Read more on how to create OAuth authentication with Connector Builder.

module.exports = {
  label : 'Connect to Twitter',
  mock_input: { access_token: ‘token’ }, //can be obtained from https://flowoauth.built.io
  oauth: 'twitter',
  input: {},
  validate : function (input, output){
    // auth data will be available in input.auth
  }
}

3. API Key

If your action/trigger requires an API Key to authenticate the user, you can use the API Key authentication method. Users can pass the API Key as a query string parameter or via an HTTP header. Below is the basic structure for the API Key authentication. You can add your custom code in this structure to create your own API Key authentication mechanism. View the template for API Key authentication or read more on how to create API key authentication with Connector Builder.

module.exports = 
{
  label:  'Connect to Demo',

  mock_input: 
              {
                "api_key" : "my api key"
              },

  input: {},

  validate: function (input, output)
  {
    // this function will be executed when running unit test cases and authData will 
    // be available in input.auth
  }

}

4. Custom

If your action/trigger requires extra information such as access token, client ID, client secret from the user along with the account credentials, you can use the custom authentication method. Given below is the basic structure for custom authentication. You can add your custom code in this structure to create your own custom authentication mechanism. View the template for custom authentication or read more on how to create custom authentication with Connector Builder.

module.exports = 
{
  label : 'Connect to Test',  
  
  mock_input: {
     "username" : "myuser",
     "apikey" : "my api key",
     "pass" : "my password"
 },

  input: 
  { 
    type: "object", 
    properties: 
    {
      //input field definition
    } 
  },

  validate : function (input, output)
  {
     // auth data will be available in input.auth
    // var username = input.auth.username
   // var password = input.auth.pass
   // var apiKey = input.auth.apikey
  }

}

Lookups

Lookup help you autofill fields with the data from your account. It retrieves a list of records associated with the specified property or account. It is important to define the lookup function in the action source code when creating the individual scaffoldings. The basic structure for creating lookup is given below. You can add your custom code in this structure to create your own lookups. Click here to read more on how to create lookups with Connector Builder.

module.exports =
{
   'name':'sample_lookup',  
   'label': 'label',

   'execute': function (input, options, output)
   {
     // your code goes here
   }

}

Example

Let’s see how this works in practice.  Suppose you want to create a new action for Github service. Follow the steps given below to achieve this:

Step 1. Install the Connector Builder globally

npm install @builtioflow/connector-builder -g

Step 2. Configure deploy key and log in to Built.io Flow

flow login

You will be prompted to enter the email and password for your Built.io Flow account.

Your Connector Builder should be installed and ready to go at this point. Next up, we'll create our first app - ‘Github'.

Step 3. Create a directory for your app with the minimum required files

flow init

You will be prompted to provide a name and description for your app which will be displayed in the ‘Custom’ tab.

Once you have entered these details, it will automatically download the required files for your app and will create an app directory inside the ‘Connector Builder’ folder. 

Step 4: Move into the newly created app directory

cd github

Step 5: Install all the libraries needed for your app

npm install

You should now have a working local app. You can start adding codes for your actions and triggers.

Step 6: Add new action

flow create action  <action_name>

Example: flow create action create_issue

flow create lookup <lookup_name>

flow create trigger <trigger_name>

Step 7: Every action/trigger needs an authentication. You can add authentication to your action/trigger using the following command: 

flow auth

Step 8: Once all the codes are added, you can test your app: 

flow test

Step 9: Deploy app 

flow deploy

Adding Custom Icon

You can also add a custom icon for your application. To do so, navigate to your application directory, and place the required icon inside the ‘icon’ folder.

The custom icon should be a square up to 128x128 pixels that is less than 1MB.

Error Validations

Built.io Flow has implemented a number of test validations for Connector Builder. They are listed in the table given below:

Downloading your App

Once you have deployed the app, you can download its ZIP file in your current directory. To do so, use this command:

flow download

When you run this command, a drop down list will appear on the screen from which you can select the app you wish to download.

Sharing/Unsharing App

You can also share your deployed app with other users: 

flow share

When you run this command, you will be prompted to enter the email address of the user with whom you want to share the app. 

To unshare, use the following command: 

flow unshare

Note: Unsharing the app will only prevent the user from creating new workflows using that app. All the existing workflows in which the app is used, will still remain active for that user.

App Versioning

Each time you update your app, a new version is created. You can get a list of all the versions of your app using this command:

flow versions

Publishing Local App to Built.io Flow

Your deployed app is always local and is only available to you and to the people you have shared it with. If you wish to publish it on to the Built.io Flow platform, you can do so by using the following command:

flow publish

Once you run this command, your app will be sent to an admin for approval. After it is approved, it will be published to the Built.io Flow platform and will be available to all users.

Adding collaborators to your app

You can also invite other users to collaborate on your app by using the following command:

 flow collaborate 

You will be prompted to enter the email ID of the user with whom you want to collaborate.

Once you have entered the email ID, the user will be added as a collaborator on your custom app. Collaborator can then access this custom app through the ‘Trigger list’ on Start icon and/or ‘Actions’ panel on the right-hand side of the canvas, depending upon the app.

Deleting the deployed action or trigger

If you want to delete any deployed action or trigger, you can do so by following the steps below:
Step 1: Download the app that contains the action or trigger you want to delete.
Step 2: Unzip the downloaded file.
Step 3: Delete the file associated with the action/trigger from the action/trigger folder and delete the name of the action/trigger from the ‘index.json’ file.
Step 4: Deploy the app using ‘flow deploy’ command.

This will remove the specified action/trigger from your app.

Creating custom connectors using Postman collections

Collections in the Postman tool, house a set of similar API requests as a group. These existing collections or set of APIs can be used to create custom connectors through the Connector Builder. 

To do so, follow the steps given below:

Step 1: Go to your Postman tool and navigate to the ‘Collections’ section.

You will see the list of existing collections. 

Step 2: Locate the collection for which you want to create custom actions. Click on the ellipsis icon (three tiny dots) given beside its name and click on ‘Export’.

Step 3: You will be prompted to specify the collection version.

Step 4: Specify the location where you want to save the exported collection. 

Note: It is recommended that you save the collection inside the app directory for ease of use.

Step 5: Open the command prompt, navigate to your app directory and enter the following command:

flow postman <collection_file_name.json>

This will create an action file for each API call included in the specified collection.

You can then redeploy your app to start using these actions like any other actions in the ‘Actions’ panel.

Help

To get the list of all the Flow-CLI commands along with their details, run the following in the command prompt:

flow help

or

flow

Given below is the list of all Connector Builder commands:

1. flow login
Configures the deploy key and logs you into Built.io Flow

2. flow logout
Logs you out from Built.io Flow by deleting your access token from the home directory.

3. flow connectors
Displays the list of all apps for the current user

4. flow init
Initializes a new Flow app in your directory. 
Example: flow init sample_app

5. flow create 
Creates a scaffolding for trigger, action, and lookup in app directory
Example: flow create trigger new_mail

6. flow auth
Creates an authentication scaffolding in the app directory

7. flow attach lookup
Attaches lookup to action or trigger field interactively

8. flow detach lookup
Detaches lookup from action or trigger field interactively

9 flow deploy
Builds app and deploys it to Built.io Flow
Connector builder will update your existing version app unless you specifically change or increase version before re-deploying it.

10. flow share
Shares your app with the specified user

11. flow unshare
Unshares the app from the specified user

12. flow download 
Downloads the ZIP file for your app to your current directory

13. flow versions
Displays the list of all versions for the current app

14. flow history
Displays the complete history of the current app

15. flow publish
Sends a publish request for the current app to Built.io Flow

16. flow import <swagger_file_path>
Converts a Swagger to an action

17. flow help or flow
Displays the list of all Connector Builder commands along with their details

18. flow collaborate
Invites the specified user to collaborate on the app

19. flow postman <collection_file_name>
Creates an action file under the action folder of your app directory for each API call in the specified collection