REST Function Continuation

Building Apps
,
1

REST services often have a limit on the number of items that can be retrieved per request. This means that to get all the items related to a particular query, the caller must make additional requests to the service, either using a URL or additional data provided by the service. The caller can use these continuation parameters to make repeated calls and retrieve all the records.

There are two types of continuation REST services. The first is a continuation URL, where the service returns a URL parameter in the response, which the caller can use to fetch the next set of items. The second type uses returned parameters to indicate the starting index or other parameters for the next call to the service.

To automate the process of making repeated calls to continuation functions, Jigx REST functions can be configured with a continuation block in the function YAML.

Examples

1.) Continuation URL

The number of records returned by Airtable is limited to 100. The response looks like this:

{
    "records": [
        {
            ...
        }
    ],
    "offset": "itrYhUckUB8OH3npJ/rec5KWtFb1rhsqc10"
}

The Jigx REST function:

provider: DATA_PROVIDER_REST
method: GET
url: https://api.airtable.com/v0/appgU2ANJz46CWrIf/Users?api_key={key}
parameters:
  key:
    ttype: secret
    location: path
    required: true
    value: APIKEY
    
outputTransform: $.{"offset":offset, "data":records.fields.{"id":id,"name":name,"order":order}}

continuation:
  parameters:
    key:
      type: secret
      location: path
      required: true
      value: APIKEY
    offset:
      type: string
      location: path
      required: true
      value: =$.offset
  url: https://api.airtable.com/v0/appgU2ANJz46CWrIf/Users?api_key={key}&offset={offset}
  when: =$.offset
records: =$.data

2.) Indexed data

This response always includes the start index and number of records to retrieve on each call. The sample of the response:

{
    "total": "634",
    "limit": "10",
    "start": "10",
    "data": [
        {
	   ...
	}
    ]	
}

The Jigx REST function:

provider: DATA_PROVIDER_REST
method: GET
url: https://developer.nps.gov/api/v1/campgrounds
records: =$.data
outputTransform: $.{'total':total,'limit':limit,'start':start,'data':data.{'id':id,'name':name,'parkCode':parkCode}}
parameters:
  x-api-key:
    location: header
    required: true
    type: secret
    value: APIKEY
  limit:
    type: string
    location: query
    required: true
    value: 10
continuation:
  when: =$number($.total) > ($number($.start) + $number($.limit))
  url: https://developer.nps.gov/api/v1/campgrounds
  parameters:
    x-api-key:
      location: header
      required: true
      type: secret
      value: APIKEY
    start:
      type: string
      location: query
      required: true
      value: =$number($.start) + $number($.limit)
    limit:
      type: string
      location: query
      required: true
      value: =$.limit
5 Likes