Time Series Functions

Limited access

This feature is only available to root and super_admin_domain profiles. Ask your administrator for proper user role profiling.

Introduction

The objective of custom time series functions is to augment the capabilities of the default functions (see Common Platform Functions) through the addition of bespoke functions written in JavaScript. These functions will be managed through the creation of a catalogue tailored to the specific requirements of each organisation.

Common Platform Functions

A common catalogue will be established for all organisations, comprising default platform time series functions. The following functions have been defined:

FIRST: Please note that the engine will store only the first received value per time bucket. Consequently, the collection engine will ignore the following values obtained in the same time bucket.
LAST: Please note that the engine will store only the last received value per time bucket, overwriting the previous ones.
AVG: The engine will calculate the arithmetic mean of all values received within the specified time interval. This feature is only available for numeric values.
MAX: The engine will save the maximum value of all received values within the configured time frame. Please note that this feature is only available for numeric values.
MIN: The engine will save the lowest value of all received values within the configured time frame. This feature is only available for numeric values.
SUM: The engine will accumulate the total of all received values within the specified time interval. This feature is only available for numeric values.
COUNT: The engine will record the total number of values received in each time bucket for subsequent analysis.
MEDIAN: The engine will calculate the median of all received values within the configured time bucket. Please note that this feature is only available for numeric values.
GEO_AVG: The engine will calculate the geometric average of all received values within the configured time bucket. Please note that this feature is only available for numeric values.
VARIANCE: The engine will calculate the variance of all received values within the configured time bucket. Please note that this feature is only available for numeric values.
STD_DEVIATION: The engine will calculate the standard deviation of all received values within the configured time bucket. Please note that this feature is only available for numeric values.

Note

Please note that these functions will be available for querying with the API defined below. However, please be aware that it will not be possible to modify or delete them.

Custom Catalog

Each organisation will have access to a custom time series functions catalogue, which will enable them to manage their functions effectively. These functions can be used to define time series columns, which can then be referenced in the aggregationFunction field. The following example illustrates this process:

{
  "name": "basic_timeserie",
  "organizationId": "organizationName",
  "description": "time series description",
  "timeBucket": 86400,
  "retention": 2592000,
  "origin": "2021-01-01T00:00:00.000Z",
  "bucketColumn": "bucket_id",
  "identifierColumn": "Admin Identifier",
  "context": [
    ...
  ],
  "columns": [
      {
          "path":"some.path.A.value._current.value",
          "name":"columnaA",
          ...
          ...
          "aggregationFunction":"AVG", //platform aggregation function
      },
      
      
      {
          "path":"some.path.B.value._current.value",
          "name":"columnaB",
          ...
          ...
          "aggregationFunction":"myCustomTimeserieFunction" //custom function
      }
  ]
}

Warning

The defined API enables the modification of the script for the custom aggregation function. It should be noted that such modifications may result in changes to the aggregated data. Consequently, there is a possibility of inconsistencies between the new aggregated data and the previous values.

Note

The defined API permits the deletion of a custom aggregation function, provided that it is not utilised in any time series.

Custom Time series Function script

Considerations when developing Custom Aggregation Function:

The following are the code’s implicit input parameters:

receivedValues: an array of new values to be used for the final value calculation.
currentValue: the column’s current value.
extra: JSON object containing the current extra variables for the column, used for the value calculation. The code should utilise implicit input values to calculate the final value and subsequently construct the result object. It will be possible to use helper functions defined in the JS API. The code must return a JSON with three specific properties:
executionResult: If the execution was completed successfully, the OK value must be returned. If not, an error description must be provided.
value: The calculated value

extra: A JSON with the updated extra variables

```json
{
  "executionResult": "OK",
  "value": 5, 
  "extra":{
    "sum": 10,
    "count": 2
  }
}
```

Note

Any datetime values included in the ‘receivedValues’ input parameter will be formatted as an ISO string.

Note

For further details on how to implement custom aggregation functions, please refer to the JavaScript API documentation, specifically the section on aggregation functions.

Functions values types

To confirm that a time series function can be assigned to a time series column, the valueTypes field will be used. This field is an array of strings that accepts the following values:

integer
number
string
boolean
date-time

These types are the same as those specified for time series column type fields. When assigning a time series function to a specific column, the system will verify that the time series function in the array matches the type specified for that column.

Note

Please note that this field is not mandatory. If it is not defined by default, it will be set to an empty array. In the event that the time series function has empty valueTypes, no validation will be carried out when assigning to a column.

Note

Please note that it will not be possible to update the time series function and remove one of the ‘valueTypes’ if the function is being used by some column whose type is the removed value. However, if all valueTypes are removed, this should not cause any issues.

Comprehensive API actions

Existing Timeseries Functions List

Getting custom timeserie functions full catalog

In both instances, only the metadata will be retrieved (no script) from both the organisations’ custom timeseries functions and the platform timeseries functions.

Usage examples

Get the full time series functions catalog (organization custom functions plus platform functions, metadata only):

curl --request GET \
     --header "X-ApiKey: <your-api-key>" \
     https://api.opengate.es/north/v80/timeseries/provision/organizations/{organizationName}/catalog

Trimmed JSON response:

[
  {
    "id": "01234567890abcdeffffffff",
    "name": "customAvg",
    "description": "Custom implementation for avg function.",
    "valueType": ["integer", "number"],
    "catalog": "ORGANIZATION"
  },
  {
    "id": "AVG",
    "name": "AVG",
    "description": "The engine will calculate the arithmetic average of all received values in the configured time bucket. Only available in numeric values.",
    "valueType": ["integer", "number"],
    "catalog": "PLATFORM"
  }
]

Create a new custom function (multipart request with a metadata JSON part and a script plain text part):

curl --request POST \
     --header "X-ApiKey: <your-api-key>" \
     --form 'metadata={"name": "customAvg", "description": "Custom implementation for avg function."};type=application/json' \
     --form 'script=@custom_avg.js;type=text/plain' \
     https://api.opengate.es/north/v80/timeseries/provision/organizations/{organizationName}/catalog

API specification

JavaScript API

Timeseries functions JS API guide

This guide describes how to write Custom Timeseries Functions and the contents of the JS API.

The API contains both the implementation of some predefined timeseries functions and some useful functions that can be used when writing Custom Aggregation Functions.

Writing Custom Aggregation Function

Input parameters

All functions will have three implicit input parameters that must be used for value calculation.

receivedValues: Array of Json of collected values. Each value will have two fields:
- value: collected value. value type depends on Column’s datastream type (please note that any datetime value will be formatted as an ISO string).
- at: datetime of collected value. This value will be defined in ISO string.
currentValue: Columns current aggregated value. The type depends on aggregation function behavior.
extra: Json with useful data for aggregated value updating. In some cases, when aggregated value must be updated, some previous auxiliary data must be used to calculate new values. The fields and their format will be defined taking into account the requirements of the function. For example, if an average data must be updated, previously received number of elements and their sum are necessary to calculate correctly new average value.

receivedValues example:

"receivedValues":[
    {
        "value": 3,
        "at": "2023-03-01T00:30:00.000Z"
    },{
        "value": 5,
        "at": "2023-03-01T01:30:00.000Z"
    },{
        "value": 6,
        "at": "2023-03-01T02:30:00.000Z"
    },{
        "value": 8,
        "at": "2023-03-01T02:30:00.000Z"
    }
];

currentValue example:

"currentValue": 4;

extra example:

"extra": {
  "sum": 8,
  "count": 2  
};

Timeserie function result

Defined function must return a result with specific format. It will be a json with two parameters:

value: New calculated value.
extra: A JSON with auxiliary data updated. The content of this JSON will be the same of the extra input parameter.

Result example:

return {
    "executionResult": "OK",
    "value": 5,
    "extra": {
      "sum": 30,
      "count": 5  
    }
};

There is an auxiliary function that takes value and extra fields as parameters and returns the json with correct format. For further description of this function check documentation.

Result example using auxiliary function:

return result.ok(5, {"sum":30}, {"count":6});

If some timeserie function execution throws an exception it will be internally caught. In this case result object will be like this:

return {
    "executionResult": "Some javascript execution error message"
}

This behavior can be forced using result.error function with an Error object or an string message. For example:

return result.error(new Error("Custom error message"));
// or
return result.error("Custom error message");

Function implementation example

Taking described Input parameters and the Result to be returned into account, AVG Aggregation Function implementation example is shown here:

var newCount = receivedValues.length;
var newSum = 0;
for(var recVal in receivedValues){
    newSum = newSum + receivedValues[recVal].value;
}
if (extra) {
    if(data.exists(extra.count)) newCount = newCount + extra.count;
    if(data.exists(extra.sum)) newSum = newSum + extra.sum;
}
var newValue = newSum / newCount;
return result.ok(newValue, {"sum":newSum}, {"count":newCount});

JS API

data.exists(value)

Aux method to check if some value has value. If value is undefined or null it will return false.

Kind: global function
Returns: Boolean - Json with result.

Param	Type	Description
value	any	value to be checked.

Example of use:

if(data.exists(receivedValues[recVal].value)){
    newSum = newSum + receivedValues[recVal].value;
}

data.undefined2null(value)

Aux method to replace all undefined fields with null values. Internally used to avoid issues when processing execution result.

Kind: global function
Returns: Any - Json with result.

Param	Type	Description
value	any	value to be checked and transformed.

Example of use:

var newValue = data.undefined2null(undefined);

console.log(newValue) // null