Webhooks, Microsoft Flow and the ArcGIS Rest API – Automating Your Web GIS

February 27, 2019

Tom Thompson

The addition of webhooks to Survey123 for ArcGIS opens a world of possibilities for improved, automated and streamlined workflows. Have you ever wondered if it was possible to use a submitted survey to update a secondary, related hosted feature service with that submitted survey’s information? Have you ever wanted to modify another hosted service when a survey was submitted automatically? Well, now you can, by leveraging Survey123 webhooks, Microsoft Flow and the ArcGIS REST API.

Survey123 for ArcGIS recently released the ability for Survey123 surveys to send information through webhooks. A webhook is a simple URL callback which provides applications with information and data when an event occurs. In this case, the event is when a survey is submitted, and the platform receiving the information is Microsoft Flow. Through webhooks and Microsoft Flow, you can leverage HTTP requests to make calls to the ArcGIS REST API to expand, automate and simplify your workflows. In this blog post, I’ll cover a workflow which updates a secondary hosted feature service with the submitted survey information, but this can quickly be expanded through other REST API operations to do more complex tasks. Let’s get started.

In addition to the details in this post, you can watch this tutorial video to learn more about automating your Web GIS using webhooks, Microsoft Flow, & the ArcGIS Rest API.

In this scenario, you have an old hosted feature service of parcel information for a city. This data has not been updated recently but is still consumed by the public in an old application. You have received complaints from city planners and the public that the parcel information is outdated. By chance, there are plans for a pilot project where city staff are inspecting water hydrants using Survey123. You want to have this new pilot project data collected in one hosted feature service and update the older parcel data hosted feature service with this recent hydrant information collected in the pilot project. Through Survey123 webhooks, Microsoft Flow and the REST API, you can accomplish both automatically.

Note: When leveraging this workflow, it is highly recommended that the survey be built using Survey123 Connect. This workflow also requires that the survey have an ID or attribute which relates the survey data to the target data. In this blog post, we’ll use Parcel ID.

Survey123 Hydrant Inspection Survey

Once the survey has been published, visit Microsoft Flow and create a connection between your survey and Microsoft Flow.

Esri’s Survey123 team released a detailed blog post on how to make a connection between your survey in Survey123 and Microsoft Flow. This is a good resource to get started in understanding how to integrate Microsoft Flow and Survey123, as well as make a connection between them.

Now that we’ve made the connection between Survey123 and Microsoft Flow, let’s begin building the workflow within Microsoft Flow.

1. If you’re working with secured services in your Web GIS, you’ll first need to generate a token so that you’re granted access to query and modify your services.

Note: If you are not working with secured services, you can skip to step 3.

To do this, add an HTTP Action to generate a token using the ArcGIS REST API’s Generate Token operation. This operation must be a POST method and will include your username, password and your referrer in the POST body. You must also configure the Content-Type headers to be application/x-www-form-urlencoded.

Microsoft Flow HTTP Action to generate a token

2. Now that you’ve made a POST request for a token, you need to parse the JSON response down into their own variables.

You can do this using the Parse JSON Data Operation. First, pass in the Get Token body into the content of the Parse JSON Data Operation and supply it with a schema. The format of the schema is as follows:

{
  "type": "object",
  "properties": {
    "token": {
      "type": "string"
    },
    "expires": {
      "type": "integer"
    },
    "ssl": {
      "type": "boolean"
    }
  }
}

Microsoft Flow Parse JSON Data Operation to parse token JSON into variables

This will parse down the response into three individual variables: ssl, token and expires, each found in the Dynamic Content dialog. You will later use the token variable to access secured services.

3. Now, find the Parcel ID you submitted in the survey and match it with the parcel stored in your Parcels hosted feature service.

Let’s use another HTTP Action and leverage the ArcGIS REST API’s Query operation to find the parcel within the Parcels feature service. Set the method in the action to GET. Find the URL to your Parcels hosted feature service REST endpoint, including the layer index, and paste it into the URI.

Next, build in the where clause parameters for the query operation. In this section, you’ll need the parcel ID from your submitted survey. In this case, the parcel ID is stored in the ‘ParcelID’ variable submitted with the survey. You must also set the returnIdsOnly to true and format the response in JSON by using &f=json. If you’re working with secured services, you’ll also have to pass the token variable to the token parameter.

Microsoft Flow HTTP Action and the ArcGIS REST API Query operation

Note: To quickly configure the Query operation parameters, visit the REST Endpoint of your service and make the query request manually, then copy the URL output from that query.

This HTTP Action will query the Parcels hosted feature service for any parcel ID which matches the Parcel ID submitted in the survey and return its ObjectID.

4. Parse down the response from the query operation.

Once again, add the Parse JSON Data Operation into your flow. The input will be the body variable from your query operation. You can use the following JSON schema.

Note: Your JSON schema may be different than below depending on which where clause parameters were used in your query.

{
  "type": "object",
  "properties": {
    "objectIdFieldName": {
      "type": "string"
    },
    "objectIds": {
      "type": "array",
      "items": {
        "type": "integer"
      }
    }
  }
}

Microsoft Flow Parse JSON Data Operation to parse query JSON into variables5.

5. Lastly, update the Parcels hosted feature service with the submitted hydrant information from your survey.

Update Features operation to update the old Parcel data with the new hydrant information. Set the method to POST and again use the URL to your Parcels hosted feature service REST endpoint, including the layer index. Also, configure the Content-Type headers to be application/x-www-form-urlencoded. Now, let’s configure the POST body.

Microsoft Flow HTTP Action and the ArcGIS REST API Update Features operation

You’ll first need to format the POST request as JSON, define the rollbackOnFailure=true (this will mean that, in the event of an error, the updates will not be pushed to the service) and pass the token variable into the body. Lastly, pass in your hydrant inspection survey information through the features attributes property. You must include the ‘OBJECTID – Item’ variable which we parsed from the query operation earlier.

Note: When doing this, Microsoft Flow will include an Apply to Each loop. This is because the ‘ObjectIDs – Item’ variable from the service is an array, and Microsoft Flow will assume the array has more than value to access, so it creates a loop.

Then, point the OBJECTID attribute to the ‘Current Item’ variable, which represents the current value in the loop (in this case, your OBJECTID). Now, include each corresponding value from the survey you want to update in the Parcels hosted feature service. These attributes must already exist in the Parcels schema. In this case, pass in the ‘HydrantCondition’ and ‘Comment’ variable values from the survey into your Parcel hosted feature service.

6. You’re finished! Save the flow and run some test data through it.

You’ve now configured the flow to generate a token to access secured services, and query the Parcels hosted feature service to find OBJECTIDs which match the ParcelID of the submitted survey. You can then use the OBJECTID to know which feature in the Parcels hosted feature service to update with the newly submitted survey information. In the end, you should have a new Hydrant Inspect dataset and updated your older, outdated Parcels dataset!

Note: Attachments are not going to be exposed in the Dynamic Content dialog. You cannot reference attachments manually either. Esri is working towards enabling access to attachments within Flow in a future release. Data from repeats will also be missing in Microsoft Flow at this time.