Defining a data set means choosing which data streams become columns. This is administration work, done once per data set.

The identifier column

Every data set needs an identifierColumn. It maps to provision.administration.identifier._current.value, with filtering enabled and sorting available, and it identifies the device each row belongs to.

Column paths

A column’s path has three parts, and the third is only required when the data stream is not a primitive value.

1. The data stream identifier — the datastreamId you want. If it contains communicationModules[], include the index of the module you mean:

device.communicationModules[0].subscription.mobile.imsi._current.value

2. The data stream field — one of:

_current.value · _current.date · _current.at · _current.feedId · _current.source · _current.sourceInfo

3. The value path — when the data stream holds an object or an array, a path down to a primitive value.

What a column can be filtered by

Every column carries a filter value that decides how queries may use it:

Value Meaning
NO Not filterable. The default
YES Optional equality filter
ALWAYS Required equality filter: every query must constrain this column
RANGE Range filter, >, < and BETWEEN, as well as equality

RANGE is only allowed on numeric columns, integer and number. Columns of type date-time are always range-searchable whatever the value says.

The sorts section

Sorting is declared in the definition, not composed at query time. The sorts section holds a list of named sorts, each an ordered list of columns with a direction, and a query asks for one by its identifier.

"sorts": [
  {
    "identifier": "sortByDeviceAsc",
    "description": "Sort by device identifier ascending",
    "columns": [
      { "name": "Prov Identifier", "direction": "ASC" }
    ]
  }
]
Field Rules
identifier Required, unique within the list. Letters, digits, spaces, _ and -. Generated as a UUID if omitted, so name it
description Optional free text
columns Required, at least one. A column name from the columns section plus ASC or DESC

At least one sort is mandatory, and the order inside columns is the sort precedence.

The reverse of every sort comes for free

For each sort you declare, OpenGate also exposes its reverse, served by the same index through a reverse scan. Those come back marked derived: true, which is read-only: the platform sets it and the web console uses it. Never declare a derived sort yourself — flipping the direction of one you already have just spends optimization units on an index you were given.

Limits

There is no fixed maximum number of filterable columns or declared sorts. Each filterable column and each declared sort consumes optimization units, and the data set has a budget of them — that budget is the limit.

Where to look What it tells you
The searchOptimizationInfo of a data set usedSearchOptimizationUnits and freeSearchOptimizationUnits
POST .../optimizationPlan What a definition would consume, before committing to it 
POST /north/v80/datasets/provision/organizations/{organizationName}/optimizationPlan

Creating

POST /north/v80/datasets/provision/organizations/{organizationName}

Updating

PUT /north/v80/datasets/provision/organizations/{organizationName}/{identifier}

Updating can affect the data already stored or the structure holding it, which starts an adaptation process. Until it completes, dirty values may be present.

These fields can be modified:

Name · Description · IdentifierColumn · Columns · Sorts

Rules for columns:

  • Names are unique. You cannot add or rename a column to a name already in use.
  • filter: ALWAYS is immutable. You cannot add or remove a column that has it, you cannot set it on an existing column, and you cannot change it away once set.
  • Paths cannot be edited. Remove the column and create it again, which gets you the same result.

The optimization unit budget applies to updates as well, so run optimizationPlan before adding filterable columns or sorts to a definition that is already close to it.

Listing and deleting

GET    /north/v80/datasets/provision/organizations/{organizationName}
GET    /north/v80/datasets/provision/organizations/{organizationName}/{identifier}
DELETE /north/v80/datasets/provision/organizations/{organizationName}/{identifier}

Read the organization’s data sets:

curl --request GET \
     --header "X-ApiKey: <your-api-key>" \
     https://api.opengate.es/north/v80/datasets/provision/organizations/{organizationName}