Connect Pipelines and Available Tasks

  • Created:
  • / Author: Rishi Shah

Digioh Connect is a powerful way to collect and move your data.  Using Pipelines, you can control what data you want and how, where and when you want it.

Within Connect, a pipeline is a set of tasks that are executed in a specific order.  These tasks allow you to access and move your data.

Each task accomplishes its own goal or goals.

Please follow the video below to review the currently available task types:

These are the currently available task types(new task types are routinely added):

Delay

This is useful for if you need to delay executing the next task or tasks in the pipleline.  With the Delay task, you configure how much time should pass before the next task in the pipeline is executed.

Http Request

This is a powerful task that will perform an Http Request to another service. This can use configured integrations for authentication, meaning if you have already established a connection to one of your services from within Connect, you will be able to leverage that connection.

With Http request, you can do any http request you would like to do.

With this task:

  • Leverage one of your already connected services by changing the “Connection” down (no need to enter the credentials again!)
  • Update the form of the Request Body using the “Request Body Type” dropdown (Form Data, JSON, text)
  • Access and use output from prior tasks by updating the “Path to Request” dropdown
  • Create output to use in other tasks
  • If you need certain headers for different endpoints, set them with the “+Header” button
  • Set the expected form of the Response Body with the “Response Body Type” dropdown (text, JSON object, JSON array, mutliple JSON objects)
  • Map parameters that will allow you to use data from other tasks to update the Http request. Example: update all “email’ within the Http request- so in the body, the url, the headers – to use the email value that is mapped in a different task within Connect!

 

Iterable API

Perform a function against the Iterable API such as updating a user, tracking a custom event, or updating a catalog item.  Within the Iterable API task you can set where to pull the data from under the various “Path to” dropdowns (Path to Email, Path to User, Path to Event, etc).  From here, you can access data from your other tasks if you need to.

Map Data

Map fields from input data into different fields as output data. Can also perform transforms on the data.

Using the Map Data Task:

  • Field Vs. Value
    • Enter a Field when using an already determed field (say form.email or form.custom_2)
      • Update the Input Field to decide where the data is pulling from – it can be Item Data or data from one of your other tasks
    • Enter a Value when setting a custom field with static text for the Input and Output
  • Output Field
    • This is where you will enter how the Output Field will be labeled
    • If you need the input field or object to be nested inside of an object in the output, use JSON syntax (for instance messages.myMessage in the output field will store the input field as key/value pair “myMessage:inputField” inside of an object – which will have

Merge Data

Sometimes you have multiple different sources of data you are mapping from and you need to get it all together into one payload to send off to your endpoint.  To do this, you can use the Merge Data task.

  • Select the path of the first source of data you’d like to merge, then click “+Path”
  • Then, select the path of the next source of data you’d like to merge with the first source
  • Do this for however many paths of data you are merging in this task, then Save

Render Template

Render Template is a powerful task that allows you to take your data and render a Handlebars, Liquid, or Scriban template.

  • Choose which data you would like to use to build the template – enter the path within that data set if necessary
  • It is recommended to use a Liquid template, however Handlebars and Scriban are also available
  • The recommended Output Type is JSON. Text is also available.
  • In the Template section, build the template you would like rendered using the data available in the “Path to Model” dropdown

Salesforce Create Object, Get Object, Query and Update Object

These are tailored solutions to take json and send it to Salesforce to update a very specific object

  • Create Object – Create an object in Salesforce
  • Get Object – Retrieve an object by Id in Salesforce
  • Query – Query Salesforce for one or more records
  • Update Object – Update an object by Id in Salesforce

Send to Pipeline

A Send to Pipeline task will send data from one pipeline to another pipeline for further processing

A Send to Pipeline task allows you to send a piece of data in the middle of execution of one pipeline into another pipeline.  Its an already built pipeline to sync widgets.  So if you are syncing a contact and you identify this contact has a widget attached to it, then you can send this widget data through a Send to Pipeline task to another pipeline.  This saves time as you will not need to do the mapping of the second pipeline twice, you can simply do it in the middle of the first pipeline, then return to the first pipeline after executing the second.

This is a great example of the flexibility of the Connect system and how you can move your data around as you need to.

  • First, get the ID of the Pipeline you would like to send the data to (this is the number listed next to the Pipeline on the Connector’s landing page)
  • Next, navigate to your “Send to Pipeline” task and edit it
  • In the JSON, update the “PipelineId” value to the ID of the pipeline you copied in the first step

Shopify

Perform a function against the Shopify API such as Get Customer By Id, Get Order, Get Product.

After choosing the API Operation from the dropdown, please note that there will be an informational box describing the function and providing any available Shopify documentation on that API call.

Snowflake Query

Query Snowflake for one or more records

Under the Connection dropdown, you will be able to see all of your previously configured Snowflake connections without any extra configuration.

Use the Parameter Mappings to update the results with the data you have already collected (for instance, you can replace any instance of userAccountId in the Snowflake result with the actual User Account Id that you have from other tasks or pipelines.

 

Task Notification

Task Notification is useful if you want to send an email at a certain point in a pipeline.  You can add a notification and it will look for a comma separated list of recipients.  To edit a Task Notification task, update the JSON with a comma separated list of recipients.

Transform Data

Transform data to fix errors or ensure proper format for destination. This is also available within Map Data, and it is recommended to use the Transform dropdown with the Map Data task when already using the Map Data task in the pipeline.

Available transforms:

  • Phone to E.164 format
  • Remove Non Alpha Characters
  • Remove Non Numeric Characters
  • Remove Html Tags
  • Convert to Lower Case
  • Convert to Upper Case
  • Convert to Title Case
  • URL Encode
  • URL Decode
  • HTML Encode
  • HTML Decode
  • Date Time and Seconds (yyyy-MM-dd hh:mm:ss)
  • Sortable Date Time with Offset ((yyyy-MM-dd hh:mm:ss +-hh:mm)
  • Date (yyyy-MM-dd)
  • Date Slashes (MM/dd/yyyy)
  • Replace
  • Prepend
  • Append
  • Convert to boolean (true/false)
  • Convert to Integer (123)
  • Convert to to Float (123.45)

Yotpo

Perform a function against the Yotpo API such as Create Or Update Customer, Create Subscriber, Product Star Rating, and more.

Leverage your already connected Yotpo integration without needing to add any extra configuration at this task creation step.