Search Automation

This will automatically do the entire pagination logic and will output a nice 'flat' list of records.

Feature Introduction

The remote search feature allows users to search for IDs by a name they're used to, e.g. they can search for a contact ID by the name of a contact:

In order for this to work, we need to setup a search, and for some connectors a pagination configuration, on the connector itself and add some fields to the ui form schema of the field where the search should take place.

Configuration on connector

The configurations on the connector are both in JSON format.

Search configuration

The search configuration contains most of the logic needed to do the search itself.

{
  "response_fields": {
    "id": "$[*].id",
    "match": "$[*].name"
  },
  "query": "{\"include\": \"{{ resource_type }}\", \"q\": \"{{ q }}\", \"per_page\": 50}",
  "type": "search",
  "endpoint": "search"
}

You can use Jinja in all fields in order to use parameters that are passed from the action or even have conditional statements involving these parameters.

response_fields

The response_fields are JSON paths to the fields that should be searched and shown in the search results.

It is always a dictionary with the keys id and match.

The id field specifies the json paths to the field that should later be inserted into the field in the connector; it is mostly an id (thus the name), e.g. the contact id in the example above).

The match field specifies the json paths to the field that the user is searching by (e.g. for the example the contact's name). For the type search it actually doesn't have any effect, however it is used for the type list.

endpoint

The endpoint is the endpoint that is used for searching by that connector and will be appended to the base domain, just like the endpoint in actions.

query

Optional

The query is a string that contains a dictionary of query parameters that will be used in the search call.

q is always the input of the field where remote search is configured in the action.

If the query parameter is given a GET call is being done by the search configuration.

body

Optional

The body is a string that contains a dictionary of request body parameters that will be used in the search call.

If the body parameter is given a POST call is being done by the search configuration.

type

The type specifies whether the search goes directly to a search endpoint of the connector, then the type is search, or whether a call to a list endpoint that returns a list of entities is done (in case the connector doesn't provide a search endpoint), then the type is list.

In case of the list type the actual search happens in our backend.

when

Optional

The, potentially rendered, result of this parameter will be compared to True and if it matches, that search configuration will be used.

This needs to be used in combination with multiple configurations.

Multiple configurations

In order to add multiple search configurations for one Connector, the configuration needs to be a list of configurations and the parameter when needs to be used, except for the last statement, which can be a 'catch all' configuration.

The when parameter will be checked from top to bottom and the one that matches first will be used. In case none match and a search configuration does not have a when parameter, that configuration will be used (i.e. similar to multiple elif and lastly an else statement).

Pagination configuration

Optional

This is only needed in case there's any pagination for the endpoint required. Parameters such as limit can be directly configured within the search configuration.

The pagination configuration is also used for Automatic Pagination Configuration

Configuration on action

On the action that uses the search some more configurations are necessary, in order to specify in which field the search should happen and to pass further parameters.

"id": {
  "title": "Account ID",
  "type": "text",
  "required": true,
  "placeholder": "Search by account name...",
  "hasRemoteSearch": true,
  "resourceType": "sales_account",
  "remoteSearchParam": [],
  "count": 10
}

The field used for search needs to be on the first level of JSON nesting (i.e. it can't be nested).

type

The type of the field has to be text

We didn't see a use case for any other field type, so we kept it simple. In case you see a use case, let us know.

placeholder

Optional

The placeholder is not required, however this is probably the best way to make it obvious to the user that they can search here.

hasRemoteSearch

hasRemoteSearch needs to be set to true.

resourceType

resourceType is required to be filled and will be accessible in the search configuration.

This could e.g. be "customer", "ticket", "user"; so an entity name that exists in the connector.

remoteSearchParam

With this field we can use input from other fields, which are also on the first level of JSON nesting and then access it within the search configuration

E.g. for Asana we need to pass the workspace_gid to the search configuration, which looks like this:

{
  "type": "search",
  "response_fields": {
    "match": "$.data[*].name",
    "id": "$.data[*].gid"
  },
  "query": "{\"resource_type\": \"{{ resource_type }}\", \"query\": \"{{ q }}\", \"count\": \"50\"}",
  "endpoint": "workspaces/{{ workspace_gid }}/typeahead"
}

The UI Form Schema in this case would look something like this:

{
  "workspace_gid": {
    "title": "Workspace GID",
    "type": "text",
    "placeholder": "Required to enable search",
    "info": "Globally unique identifier for the workspace or organization."
  },
  "project_id": {
    "title": "Project ID",
    "type": "text",
    "required": true,
    "info": "Globally unique identifier for the project.",
    "placeholder": "Search by project name…",
    "hasRemoteSearch": true,
    "resourceType": "project",
    "remoteSearchParam": [
      "workspace_gid"
    ]
  }
}

count

Optional

This could e.g. be 5 or 10; and specifies the number of entities that we want to return (in case this is configured in the search configuration).

We haven't seen a use case yet where it makes sense to configure this on the action. If we don't see one in the next month, we can probably remove it entirely

Examples

Freshsales - Search

Search configuration

{
  "type": "search",
  "response_fields": {
    "match": "$[*].name",
    "id": "$[*].id"
  },
  "query": "{\"include\": \"{{ resource_type }}\", \"q\": \"{{ q }}\", \"per_page\": 50}",
  "endpoint": "search"
}

Action search configuration

"id": {
  "title": "Account ID",
  "type": "text",
  "required": true,
  "placeholder": "Search by account name...",
  "hasRemoteSearch": true,
  "resourceType": "sales_account",
  "remoteSearchParam": [],
  "count": 10
}

HubSpot - Search

Search configuration

HubSpot has a search endpoint, which works great for the resource types that could have potentially thousands of entries (i.e. using type list is not an option), however, it does not work for all resource types and for the ones that are not supported get calls need to be made.

Thus, multiple configurations are needed in order to cover all resource types:

  1. For the search endpoint HubSpot expects a POST request, thus we need to provide the field body instead of the field query. This configuration will use the when parameter, as HubSpot has a set list of resource types that are supported by this endpoint.

  2. For the get endpoints, search type list needs to be used. Here, the when parameter will not be used, as it should always be used if the resource type is not supported by the search endpoint (and thus the first search configuration).

[
  {
    "when": "{{ resource_type == 'companies' or resource_type == 'contacts' or resource_type == 'deals' or resource_type == 'feedback_submissions' or resource_type == 'products' or resource_type == 'tickets' or resource_type == 'line_items' or resource_type == 'quotes' }}",
    "type": "search",
    "body": "{ \"filterGroups\": [{\"filters\": [{\"propertyName\": \"{% if resource_type == 'companies' %}name{% elif resource_type == 'contacts' %}lastname{% elif resource_type == 'deals' %}dealname{% endif %}\", \"operator\": \"CONTAINS_TOKEN\", \"value\": \"{{ q }}\"}]}]}",
    "response_fields": {
      "id": "$.results[*].id",
      "match": "$.results[*].properties.{% if resource_type == 'companies' %}name{% elif resource_type == 'contacts' %}lastname{% elif resource_type == 'deals' %}dealname{% endif %}"
    },
    "endpoint": "crm/v3/objects/{{ resource_type }}/search"
  },
  {
    "type": "list",
    "query": "",
    "response_fields": {
      "id": "$.results[*].id",
      "match": "$.results[*].lastName"
    },
    "endpoint": "crm/v3/owners"
  }
]

Action search configuration

"companyId": {
  "type": "text",
  "title": "Company ID",
  "info": "Company to be updated",
  "required": true,
  "placeholder": "Search by company name…",
  "hasRemoteSearch": true,
  "resourceType": "companies",
  "remoteSearchParam": []
}

Slack - List

Search configuration

This is not dynamic yet, needs to be improved

{
  "type": "list",
  "response_fields": {
    "match": "$.channels[*].name",
    "id": "$.channels[*].name"
  },
  "endpoint": "conversations.list"
}

Action search configuration

"channel": {
  "title": "Channel Name without the # ",
  "type": "text",
  "placeholder": "marketing-team",
  "required": true,
  "info": "Search by name...",
  "hasRemoteSearch": true,
  "resourceType": "channels",
  "remoteSearchParam": []
}

Stripe - List

Search configuration

Here we use conditional Jinja logic in order to reference to the proper match field.

{
  "endpoint": "{{ resource_type }}",
  "type": "list",
  "response_fields": {
    "id": "$.data[*].id",
    "match": "$.data[*].{% if resource_type == 'customers' or 'products' %}name{% elif resource_type == 'invoices' %}customer_name{% endif %}"
  }
}

Action search configuration

"id": {
  "type": "text",
  "title": "ID",
  "required": true,
  "info": "Search by name...",
  "hasRemoteSearch": true,
  "resourceType": "customers",
  "remoteSearchParam": []
}
PreviousConnector Auth ValidationNextPagination Automation

Last updated