Training plans

Limited access

This feature is only available to root and super_admin_domain profiles. Ask your administrator for proper user role profiling.

What a training plan is

A training plan is a packaged answer to one question — is this RADIUS session unusual for its APN?, is this part defective? — built and validated by the platform team and published in a catalogue. It bundles:

  • the algorithm and the whole training recipe: how the data is cleaned, split, transformed, trained and evaluated;
  • the container image that runs that recipe;
  • the data it needs: which source types it accepts (a file, a time series, or either) and, for time series, the columns it expects to find;
  • the configuration you must supply when you create a trainer — an APN, for example;
  • the minimum amount of data below which a training is refused rather than producing a meaningless model;
  • and, implicitly, the inference contract of the model it produces and the rule it creates to call it.

You do not modify a plan. You create a trainer that runs it, and the plan’s own pages below tell you what to feed it and what comes out.

Listing the catalogue

curl --request GET \
     --header "X-ApiKey: <your-api-key>" \
     https://api.opengate.es/ai/trainingPlans

The catalogue is platform-wide, not per organization, and read-only through the API. Each entry:

{
  "identifier": "6f1c2a4e-…",
  "name": "RADIUS anomalies per APN (Isolation Forest)",
  "description": "Detects anomalous data sessions of the subscriptions of one APN",
  "modelType": "anomaly",
  "modelFormat": "isolation-forest",
  "source": ["file", "timeserie"],
  "configFields": ["apn"],
  "columnData": [
    { "name": "APN", "type": "STRING", "description": "Access point name of the session" },
    { "name": "sbytes", "type": "LONG", "description": "Bytes sent by the device" }
  ],
  "minDataToTrain": { "value": 15000, "unit": "ITEMS" },
  "image": { "name": "trainingplan-if-radius-anomalies-per-apn", "tag": "1.0.0" }
}
Field Meaning How you use it
identifier The plan’s id imageExecution.trainingPlan.identifier when creating a trainer
name, description What the plan does, for people The console shows them in the plan picker
modelType The family of problem: anomaly, classification… Groups plans in the console
modelFormat The algorithm: isolation-forest, autoencoder, pytorch… Informative; it also names the registered model
source The data source types the plan accepts: file, timeserie or both Decides whether source.path or source.timeserie is allowed in the trainer
configFields The configuration keys the plan needs Every one of them must appear in imageExecution.configuration
columnData The columns the plan expects in its input For a time-series source, map each of them to a column of your time series
minDataToTrain The minimum amount of data: a number of ITEMS (rows, images per class) or of DAYS Below it, the training fails with an explicit message instead of producing a bad model
image The container image and tag the scheduler runs Informative

The plans available

  • RADIUS session anomalies — Isolation Forest

    Learns what a normal data session looks like on one APN and flags the sessions that do not fit: the data it needs, how it trains, the prediction request and answer, and the rule and alarm it creates.

  • RADIUS session anomalies — Autoencoder

    The same RADIUS-per-APN problem solved with a neural network that learns to reconstruct normal sessions and flags the ones it reconstructs badly: data, training, prediction contract and the rule it creates.

  • Image anomaly detection

    Tells defective items from correct ones in photographs and draws a heat map of the defect: the folders of images it trains on, the two models it combines, the prediction request over the organization's file space, and the rule and alarm it creates.

The three plans share the same lifecycle, described in How it works: what differs between them is the data they take, the request their inferencer answers, and the rule they create.

RADIUS · Isolation Forest RADIUS · Autoencoder Image anomaly detection
Source types file, time series file, time series file (a folder)
Input RADIUS session records of one APN RADIUS session records of one APN Photos in correct/ and incorrect/ folders
Configuration apn apn —
Inference request sbytes, dbytes, spkts, dpkts, dur sbytes, dbytes, spkts, dpkts, dur image_route, generate_heat_map
Inference answer prediction, anomaly_score, explanation prediction, anomaly_score, explanation predictions, anomaly_score, heatmap_path
Rule triggers on GPRS presence turning STOP GPRS presence turning STOP A new imagePathToCheck value
Alarm deviceWithAnomaly deviceWithAnomaly imageWithAnomaly

API specification

Subsections of Training plans

RADIUS session anomalies — Isolation Forest

What it detects

Every data session a SIM subscription opens through a mobile operator is recorded by RADIUS accounting: how long it lasted, how many bytes and packets went each way. On a given APN those sessions have a shape — an IoT fleet sending small periodic uploads looks nothing like a fleet streaming video. This plan learns that shape and scores each new session against it, so that a SIM that suddenly uploads gigabytes, or holds a session open for days, stands out.

The algorithm is an Isolation Forest: an ensemble of random trees where points that are easy to isolate — few splits away from everything else — are anomalies. It needs no labelled examples of bad sessions, only enough normal traffic to learn from.

Data it needs

The plan reads RADIUS session records and keeps the sessions of one APN, given as the apn configuration field. Each record needs these columns; with a time-series source you map them in the trainer, with a file source they must be present under these names in a Parquet file:

Column Type Meaning
APN string Access point name of the session
SessionState string State of the session record
IP_GGSN, IP_Device string Gateway and device IP addresses
sbytes, dbytes integer Bytes sent and received by the device
spkts, dpkts integer Packets sent and received
dur integer Duration of the session

Records with missing values are dropped, and so are TERMINATED records that carry no traffic counters at all. The plan refuses to train when the APN has fewer sessions than the catalogue’s minDataToTrain (15 000 when the plan does not say otherwise): the execution fails with Not enough data for APN rather than register a model nobody should trust.

How it trains

From the five counters the plan derives eight more features — totals, ratios between directions, rates per second of duration — and scales them robustly to the [0, 1] range. The data is split 75 / 12.5 / 12.5 into training, validation and test. After fitting, the forest scores the training set and the plan sets the decision threshold at the 95th percentile of those scores: anything scoring above it at inference time is an anomaly. Two numbers are recorded with the model version and shown as its metrics:

Metric Meaning
calculated_threshold The score above which a session is called anomalous
training_max_score The highest score seen in training; inference scores are divided by it so anomaly_score reads as a fraction

The prediction request

The inferencer answers POST /api/predict with the five counters of one session:

{ "sbytes": 18234, "dbytes": 1203991, "spkts": 210, "dpkts": 980, "dur": 3600 }
{
  "prediction": 1,
  "anomaly_score": 1.18,
  "explanation": [
    { "dbytes": 0.071, "received_bytes_rate": 0.044, "total_bytes": 0.031 }
  ]
}
Field Meaning
prediction 1 anomalous, 0 normal
anomaly_score The isolation score divided by the training maximum. Above roughly 1 means more extreme than anything seen in training
explanation Only when prediction is 1: the features that pushed the score up, with their contribution, computed with SHAP on the forest. Empty otherwise

The rule it creates

Unless the trainer says createRule: false, the first successful training creates an ADVANCED rule named after the model in default_channel, inactive until the inferencer is activated. What it does:

  1. Triggers on device.communicationModules[].subscription.mobile.presence.gprs, and acts only when the value is STOP — the session has just closed — and the subscription’s session record belongs to the configured apn.

  2. Sends that session’s sentBytes, receivedBytes, sentPackets, receivedPackets and duration to the inferencer.

  3. Collects three datastreams on the entity, dated at the session’s timestamp:

    Datastream Value
    withAnomaly true or false
    score The anomaly_score
    explanation The explanation as text, when the rule parameter shouldShowAnomalyReason is true

  4. Opens the alarm deviceWithAnomaly (severity URGENT, priority MEDIUM) when the entity becomes anomalous and was not before, carrying the explanation as extra information; closes it when a later session comes back normal.

The rule’s parameters — inferenceServiceURL, apn, shouldShowAnomalyReason — are editable in the rules editor, as is the whole script. The datastreams it collects must exist in the entity’s data model.

Choosing between this plan and the Autoencoder

Both plans take the same data and answer the same request, so a rule written for one works for the other. The Isolation Forest trains in seconds, needs no tuning and explains its decisions feature by feature; it is the one to start with. The Autoencoder is worth a try when the forest flags too much or too little and you have plenty of data: it learns a smoother notion of normal at the price of a longer training.

RADIUS session anomalies — Autoencoder

What it detects

The same thing as the Isolation Forest plan: data sessions of the SIM subscriptions of one APN whose traffic profile does not match the APN’s usual behaviour. What changes is how usual is learned.

An autoencoder is a neural network trained to compress each session into a few numbers and rebuild it from them. It only sees normal traffic while training, so it becomes good at rebuilding normal sessions — and bad at rebuilding anything else. The reconstruction error of a new session is its anomaly score.

Data it needs

Identical to the Isolation Forest plan: RADIUS session records with APN, SessionState, IP_GGSN, IP_Device, sbytes, dbytes, spkts, dpkts and dur, filtered to the configured apn, with the same cleaning and the same minDataToTrain check. A time series that feeds one plan feeds the other unchanged.

How it trains

The same thirteen features and scaling as the forest, then a symmetric network of five hidden layers (60 · 30 · 25 · 30 · 60 neurons) trained with the Adam optimiser on mean squared error, with early stopping and L2 regularisation. The decision threshold is again the 95th percentile of the training reconstruction errors, and the same two metrics are recorded:

Metric Meaning
calculated_threshold The reconstruction error above which a session is called anomalous
training_max_score The highest error seen in training, used to normalise anomaly_score

Training a network takes longer than growing a forest. Give the trainer a generous execution timeout.

The prediction request

Same request, same answer as the forest:

{ "sbytes": 18234, "dbytes": 1203991, "spkts": 210, "dpkts": 980, "dur": 3600 }
{
  "prediction": 1,
  "anomaly_score": 1.42,
  "explanation": [
    { "dbytes": 0.213, "received_bytes_rate": 0.187, "total_bytes": 0.171 }
  ]
}

anomaly_score is the reconstruction error divided by the training maximum. explanation, present only for anomalies, lists the features whose individual reconstruction error is in the top quarter — the parts of the session the network could not make sense of.

The rule it creates

The same rule as the Isolation Forest plan: triggered by the GPRS presence turning STOP on a subscription of the configured APN, collecting withAnomaly, score and explanation, and opening and closing the deviceWithAnomaly alarm. Because both plans share the request and the answer, an organization can train both on the same data and compare them side by side, each with its own model name.

Image anomaly detection

What it detects

Given a photograph of an item — a part on a line, a meter, a connector — the model says whether it looks like the correct examples it was trained on or like the incorrect ones, and when it finds a defect it produces a heat map: the same image with the suspicious region painted over, saved next to the original.

Data it needs

This plan takes a file source only: a folder in your organization’s file space with two sub-folders of .jpg, .jpeg or .png images:

<your folder>/
  correct/      photographs of items that are fine
  incorrect/    photographs of items with the defect

Upload the images — a .zip or .tar.gz is extracted on arrival — and give the trainer the folder as source.path. Both classes need at least minDataToTrain images each; the training fails with a message naming the class that falls short. The more varied the correct set, the fewer false alarms.

How it trains

Two models are trained and used together:

  • A ResNet-18 classifier, pre-trained on ImageNet and fine-tuned on your two folders to output the probability that an image is defective. Images are resized to 224 × 224 and lightly jittered in brightness and contrast so the model does not learn the lighting of your photo booth.
  • A PaDiM anomaly model on the activations of one of the network’s inner layers, which estimates how far each region of a new image is from the distribution of correct images — this is what the heat map comes from, and it catches defects the classifier has never seen.

The plan’s metric is the classifier’s F1 score on the test split; it is recorded with the version.

The prediction request

The inferencer answers POST /api/predict with the path of an image relative to the organization’s file space — the same space the file connector manages, mounted for the inferencer at /data:

{ "image_route": "/line-3/2026-09-03/part-0412.jpg", "generate_heat_map": true }
{
  "predictions": 1,
  "anomaly_score": 0.87,
  "heatmap_path": "/line-3/2026-09-03/part-0412_heatmap.jpg"
}
Field Meaning
predictions 1 defective, 0 correct
anomaly_score Between 0 and 1. The classifier’s probability when it fires; otherwise PaDiM’s distance mapped onto the same range
heatmap_path When the item is defective and generate_heat_map was not false: the heat map written next to the original as <name>_heatmap.<ext>. null otherwise

An image_route that does not exist returns 422.

The flow is: the classifier decides first; if it sees a defect the answer is its probability and a Grad-CAM heat map of what it looked at. If it sees nothing, PaDiM gets a second look and can still call the item defective when its distance exceeds the plan’s minimum, with its own heat map.

The rule it creates

The rule named after the model, in default_channel, inactive until the inferencer is activated:

  1. Triggers on the datastream imagePathToCheck — collect the path of a new photograph into it, and the rule runs.
  2. Sends that path to the inferencer, with generate_heat_map taken from the rule parameter requestHeatMap.
  3. Collects imageWithAnomaly (true / false) and, when there is one, imageWithHeatMapPath, dated at the photograph’s timestamp.
  4. Opens the alarm imageWithAnomaly (severity URGENT, priority MEDIUM) naming the image when the entity becomes anomalous, and closes it when a later image is correct.

So a camera integration only has to do two things: drop the photograph into the organization’s file space and collect its path into imagePathToCheck. The rest is the loop described in How it works.