The search API uses the following filtering options to facilitate the search and allow to perform a wide range of consultations.

Several techniques solve the filtering issue when you’re querying over a RESTful interface. For example, you can use standard HTTP parameters to add filtering capabilities to your query. It’s pretty simple but doesn’t cover complex needs. We require a SQL-like approach, with typical operators like AND, OR, EQUAL, NOT EQUAL, etc. OpenGate allows you to filter your queries by sending a POST request to a specific URI. In the POST request, you must send a JSON document with a fashionable DSL structure. It is a command pattern approach in contrast with the entity/collection pattern used in the provisioning API.

Filtering operators

Filtering comparison operator list

• eq: Equals.
• neq: Not equals.
• like: Regex pattern like.
• gt: Greater than.
• lt: Lower than.
• gte: Greater than or equals.
• lte: Lower than or equals.
• in[]: Included in a concrete group.
• nin[]: Not included in a concrete group.
• exists: Exists.
• within: Included in an areas.geometry GeoJson (exclusive for Area search).

Filtering logical operators list

• and[]: And
• or[]: Or

How to use filters

Let’s suppose we want to filter devices with device.operationalStatus equals to NORMAL and with device.communicationModules[].mobile.imei starting with 351873000102290.

If we were dealing with a SQL database we’d write the following SQL sentence:

SELECT * FROM device
WHERE device.operationalStatus LIKE 'NORMAL'
AND device.communicationModules[].mobile.imei LIKE '351873000102290'

Translating the previous SQL sentence to OpenGate searching API we’ll have:

{
  "filter": {
    "and": [
      {
        "like": {
          "provision.device.administrativeState": "NORMAL"
        }
      },
      {
        // The result will contain all devices with collected operational Status that
        // contains NORMAL and are related with communications modules with collected
        // imei containing 351873000102290
        "like": {
          "provision.device.communicationModules[].mobile.imei": "351873000102290"
        }
      }
    ]
  }
}

The result will contain all devices with collected operational Status that contains NORMAL and are related to communications modules with collected imei containing 351873000102290.

Another example comparing SQL to JSON, searching all devices except the one with serialNumber equal to 82A75D494B0EBF7A95587285AE78E83F:

SELECT * FROM device WHERE serialNumber <> '82A75D494B0EBF7A95587285AE78E83F'
/north/v80/search/devices

{
  "filter": {
    "neq": { "device.serialNumber": "82A75D494B0EBF7A95587285AE78E83F" }
  },
  "limit": {
    "start": 2
  }
}

More filtering examples

Click on any of the following links to expand the contents.

{
  "filter": {
    "and": [
      { "in": { "device.name": ["device_1", "device_2"] } },
      { "nin": { "device.description": ["device default description"] } }
    ]
  },
  "limit": {
    "start": 26,
    "size": 50
  }
}

{
  "filter": {
    "eq": { "tag": "my_sticky_tag" }
  },
  "limit": {
    "start": 26,
    "size": 50
  }
}

{
  "filter": {
    "and": [
      { "like": { "device.model": "EF5" } },
      { "exists": { "device.identifier": true } }
    ]
  },
  "limit": {
    "start": 1,
    "size": 10
  }
}

{
  "filter": {
    "like": { "device.software": "1.0" }
  },
  "limit": {
    "start": 1,
    "size": 10
  }
}

{
  "filter": {
    "in": { "operationEntityType": ["SUBSCRIPTION", "ASSET"] }
  },
  "limit": {
    "start": 1,
    "size": 10
  }
}

{
  "filter": {
    "in": { "device.name": ["DEVICE22"] }
  },
  "limit": {
    "start": 1,
    "size": 10
  }
}

{
  "filter": {
    "like": {
      "provision.device.location._current.value.postal": "41015"
    }
  }
}

By default, the search response includes all the data streams of the searched entities. You can retrieve only the information you need using the select sub-document in the search JSON.

The select sub-document can be used only on entity searching and must not be empty.

You can also use this sub-document when you search for information in CSV format.

If the select clause isn’t in the filter, the behavior is the following:

• In JSON format, the response will contain all the data streams collected or provisioned in the devices you are searching.
• In CSV format, the search API raises an error in the response, asking for the select clause.

As described above, any data stream of the default data models or data models defined by the user can be used as select fields.

The order to apply the filters is securitization and next the following fields whenever there are resourceType, sort, filter, select (the data streams to show)

Select JSON object

• select[]: Array of parameters to be selected.

  • name: String. Data stream name in the default or user-defined data models.

  • fields[]: Array of strings with the name of the fields to be retrieved.

    The possible values are: (See current object attributes table for field description):

    • value
    • date
    • at
    • from
    • tags
    • feedId
    • scoring.performance
    • scoring.qrating
    • provType
    • value.simplexAttribute: where simplexAttribute is an attribute of the complex object. For example, the provision.device.location is a complex data stream. If you need only de postal code, the value would be value.postal

  • alias: String. Shortname replaces the parameter’s full name when a CSV format is required. Example:

    • Using “alias”=“imei”
    • The device.communicationModules[].mobile.imei becomes imei in the CSV header

    The complete data stream name in the CSV header will appear if this field doesn’t exist. The CSV format shows this field, but the JSON format ignores it.

Select examples

Here’s how to search devices with a filter with a select clause

The following snippet shows the request using curl:

curl --request POST \
     --data-binary @subscription.json \
     --header "X-ApiKey: YOUR_API_KEY_HERE" \
     --verbose \
     https://[your_opengate_address]/north/v80/search/devices \
     -H "Content-type: application/json"

You include the filter with the select clause in the request’s body.

Filter using predefined device fields

{
  "select": [
    {
      "name": "provision.device.identifier",
      "fields": [
        {
          "field": "value",
          "alias": "id"
        },
        {
          "field": "date",
          "alias": "date"
        }
      ]
    }
  ]
}

Filter obtaining a field of a complex value

{
  "select": [
    {
      "name": "provision.device.location",
      "fields": [
        {
          "field": "value.postal",
          "alias": "Postal code"
        }
      ]
    }
  ]
}

Depending on the parameter header, the response can be in two different formats, see HTTP Header Options:

• JSON Format (Default)
• CSV Format

Response to the filter in JSON format example

{
  "devices": [
    {
      "id": "bca8cbbb-b151-442b-8b0d-96ed77789c45",
      "device": {
        "operationalStatus": {
          "current": {
            "value": "NORMAL",
            "date": "2017-02-02T09:05:58Z",
            "performance": 75
          }
        },
        "communicationsModules": [
          {
            "identifier": "351873000102290",
            "mobile": {
              "imei": {
                "current": {
                  "value": "351873000102290",
                  "date": "2017-02-02T09:05:58Z"
                }
              }
            }
          }
        ]
      },
      "health": {
        "heart": {
          "rate": {
            "current": {
              "value": 60,
              "date": "2017-02-02T09:05:58Z",
              "performance": 78
            }
          }
        }
      }
    }
  ]
}

Response to the Filter in CSV format example

device.operationalStatus.current.value;device.operationalStatus.current.updated;device.operationalStatus.current.performance;imei.current.value;imei.current.updated;imei.current.performance;rate.current.value;rate.current.updated;rate.current.performance;
"NORMAL";"2017-02-02T09:05:58Z";75;"351873000102290";"2017-02-02T09:05:58Z";;60;"2017-02-02T09:05:58Z";78

The searching API allows ordering of the results. You can sort your search results, including an order object to the JSON.

Quick sort example

{
  "filter": {},
  "sort": {
    // Array of ordering parameters
    "parameters": [
      {
        // Name of parameter defined in the Default Datamodels
        "name": "provision.administration.identifier",
        // Enumeration string with ordering type. Valid values are:
        // - ASCENDING
        // - DESCENDING
        "type": "ASCENDING"
      }
    ]
  }
}

Thanks to the group keyword in the search JSON, it is possible to group by some parameters of the default data models. This keyword is optional.

Quick grouping example:

{
  "filter": {},
  // Group sub-document
  "group": {
    // List of data streams to group by
    "parameters": [
      {
        "name": "provision.device.model"
      }
    ]
  }
}

Group examples

Click on any of the following links to expand the contents.

{
  "filter": {
    "and": [
      {
        "like": {
          "provision.administration.organization": "battery_organization"
        }
      }
    ]
  },
  "group": {
    "parameters": [
      {
        "name": "provision.device.model"
      }
    ]
  }
}

{
  "summary": {
    "count": 6,
    "summaryGroup": [
      {
        "provision.administration.organization": {
          "count": 6,
          "list": [
            {
              "count": 6,
              "name": "battery_organization"
            }
          ]
        }
      },
      {
        "provision.administration.channel": {
          "count": 6,
          "list": [
            {
              "count": 6,
              "name": "battery_channel"
            }
          ]
        }
      },
      {
        "provision.device.model.name": {
          "count": 6,
          "list": [
            {
              "count": 1,
              "name": "model_name-7"
            },
            {
              "count": 2,
              "name": "model_name-4"
            },
            {
              "count": 1,
              "name": "model_name-10"
            },
            {
              "count": 1,
              "name": "model_name-5"
            },
            {
              "count": 1,
              "name": "model_name-11"
            }
          ]
        }
      }
    ]
  }
}

The API allows you obtaining the response to a search in blocks with predefined number of results.

• limit:
  • start: Page number you request. The count starts with the number 1
  • size: The number of entities that you can see on the page

Paginated examples

Paginated example response

{
    "page" : { "number" : 1 },
    "resources" : [
        { ... },
        { ... },
        { ... },
        { ... },
        ...
        { ... }
    ]
}

Warning about the previous example

The number attribute is the current number of pages based on the limit setup.

What can you do to get the following page? It’s easy. You only have to include a limit object in your query. See next example.

Pagination example request

{
  "limit": {
    "start": 2
  }
}

Paginated example generic response

{
   "page" : { "number" : 2 },
   "resources" : [
       { ... },
       { ... },
       { ... },
       { ... },
       ...
       { ... }
   ]
}

See previous warning about the resources word in the example.

You can change the page limit from the beginning. Supposing you want to retrieve 50 items per query, you must set up the limit object with a starting point and the page size you want.

Paginated example request

Changing the starting page and the limit

{
  "limit": {
    "start": 2,
    "size": 50
  }
}

The top margin for the page size in the limit object is 1000. You’ll receive a server error response if you set up a size attribute over this limit.

Another example of pagination response

With the starting page and the limit changed

{
    "page" : { "number" : 2},
    "resources" : [
         { ... },
         { ... },
         { ... },
         { ... },
         ...
         { ... }
    ]
}

See previous warning about the resources word in the example.

Responses to all search requests include a summary object with different counters regarding the results obtained. It is closely related to the grouping feature.

By default, the summary always shows the total count, the organization’s grouping counter, and the channel grouping counter.

• count (field): number of occurrences found in the whole search
• summaryGroup []: array of type of summarized specific object structure
  • SpecificObjectParameterDatamodel: object inside the Parameter of the data models
    • count: number of these specific elements found
    • list: array of each type of summarized element
      • count: number of these specific elements found
      • name: value of the parameter of the data model

Here’s how to search devices with a summary without a group clause

This is the request using curl:

curl --request POST \
     --data-binary @subscription.json \
     --header "X-ApiKey: YOUR_API_KEY_HERE" \
     --verbose \
     https://[your_opengate_address]/north/v80/search/devices/summary \
     -H "Content-type: application/json"

Some grouping examples

Response summary without grouping

{
  "summary": {
    "count": 6,
    "summaryGroup": [
      {
        "provision.administration.organization": {
          "count": 6,
          "list": [
            {
              "count": 6,
              "name": "organization1"
            }
          ]
        }
      },
      {
        "provision.administration.channel": {
          "count": 6,
          "list": [
            {
              "count": 6,
              "name": "channel1"
            }
          ]
        }
      }
    ]
  }
}

This is the request using curl:

curl --request POST \
     --data-binary @subscription.json \
     --header "X-ApiKey: YOUR_API_KEY_HERE" \
     --verbose \
     https://[your_opengate_address]/north/v80/search/devices/summary \
     -H "Content-type: application/json"

In the body of the request, you include the filter with the select clause.

Example Filter using group clause

{
  "filter": {
    "and": [
      {
        "like": {
          "provision.administration.organization": "battery_organization"
        }
      }
    ]
  },
  "group": {
    "parameters": [
      {
        "name": "provision.device.model"
      }
    ]
  }
}

Response summary with group clause

{
  "summary": {
    "count": 6,
    "summaryGroup": [
      {
        "provision.administration.organization": {
          "count": 6,
          "list": [
            {
              "count": 6,
              "name": "battery_organization"
            }
          ]
        }
      },
      {
        "provision.administration.channel": {
          "count": 6,
          "list": [
            {
              "count": 6,
              "name": "battery_channel"
            }
          ]
        }
      },
      {
        "provision.device.model.name": {
          "count": 6,
          "list": [
            {
              "count": 1,
              "name": "model_name-7"
            },
            {
              "count": 2,
              "name": "model_name-4"
            },
            {
              "count": 1,
              "name": "model_name-10"
            },
            {
              "count": 1,
              "name": "model_name-5"
            },
            {
              "count": 1,
              "name": "model_name-11"
            }
          ]
        }
      }
    ]
  }
}