How it works
The whole loop in one picture
Everything in this section is a step of the loop below. The names in bold are the API resources you will meet on the following pages.
sequenceDiagram
participant U as You
participant TA as Trainers API
participant SC as Scheduler
participant J as Training job
participant IA as Inferencers API
participant RU as Rules engine
U->>TA: POST trainer (plan, data source, schedule)
TA->>SC: schedule image execution or pipeline
SC-->>J: at the scheduled time: export data, run the plan's image
J->>J: train, evaluate, register the model version
J->>J: build the inference image and push it
J->>IA: create inferencer, or add the new version to it
J->>RU: create the rule (first time only)
J-->>SC: callback: OK
U->>IA: activate a version
IA->>RU: deploy the service, activate the rule
RU->>IA: on each matching reading: POST prediction request
IA-->>RU: prediction
RU->>RU: collect datastreams, open or close alarms
1. You create a trainer
A trainer is a request to run a training plan on a data source, optionally on a schedule
(POST /ai/organization/{organizationId}/trainer, see Trainers). You give it:
- the plan to run, by its identifier from the catalogue;
- the model name — lowercase letters, digits and hyphens. It becomes the name of the model, of the inferencer, of the rule and of the scheduler entry, so choose it with care: it cannot be changed later and only one trainer per model name can exist in an organization;
- the plan’s configuration — the fields the plan declares, such as the APN to learn;
- a data source — a file in your organization’s file space, or a time series to export;
- a schedule, if the model has to be retrained periodically; and
- an execution timeout for the training job.
The Trainers API checks the plan exists, stores a small Kubernetes secret with your API key and organization so the job can call the platform back on your behalf, and hands the work to the scheduler.
2. The scheduler runs it
The scheduler is a general-purpose service: it runs REST requests, container images and pipelines of both on a cron expression or an interval, and keeps a history of every execution. A trainer becomes one of two things there, both named after the model:
| Data source | What is scheduled | Steps |
|---|---|---|
A file (source.path) |
An image execution | Run the plan’s container with dataSourcePath=/data/<your path> |
A time series (source.timeserie) |
A pipeline | 1. POST the time series Parquet export, writing <model>-<plan>-<timeseries>.parquet into your file space2. Run the plan’s container with dataSourcePath pointing at that file |
Inside the job your organization’s file space is mounted at /data, which is why every path the plan sees
starts there. The job also receives the plan’s configuration as environment variables, the platform-wide AI
settings from a shared secret, and a callbackUri it must call when it finishes. The training job is a
Kubernetes Job with no retries: it either completes, fails, or is killed when the timeout expires.
3. The job trains, versions and publishes
Every plan image runs the same three commands, provided by the training template framework:
- Generate the run profile — experiment
<organization>-<model>, registered model<organization>-<model>-<algorithm>, data location fromdataSourcePath. - Run the recipe — ingest, split, transform, train, evaluate, register. Each run is tracked in the platform’s MLflow, which is where model versions come from: the first successful training registers version 1, a retraining registers version 2, and so on. If the data does not meet the plan’s minimum, the run fails here and says so.
- Publish the inference — download the latest run’s model and metrics, wrap them in the framework’s
FastAPI inference server, build a container image named
<organization>-<model>:v<version>and push it to the platform registry. Then, through the Inferencers API:- if an inferencer named after the model does not exist yet, create it with that image as its only
version — and, unless the trainer was created with
createRule: false, first create the plan’s rule indefault_channeland link it to the inferencer; - if it already exists, add the image as a new version.
- if an inferencer named after the model does not exist yet, create it with that image as its only
version — and, unless the trainer was created with
Whatever happens, the job reports back to the scheduler with OK, ERROR or — if it was killed by the timeout —
TIMEOUT, and a description. That report is what you see as the execution’s history.
4. You activate a version
A freshly created inferencer has one version and nothing deployed. Activation is your call
(PUT .../inferencer/{inferencerId}/activation?image=<version>&active=true), because it is the moment the
platform starts spending resources on your behalf:
- the Inferencers API deploys the image as a service reachable inside the platform at
https://<organization>-<model>:8443/api/predict— lowercase, underscores turned into hyphens; - it waits for the container to be running;
- it sets the linked rules to
active: true.
You can activate a specific version, or latest: then every new version a retraining produces replaces the
running one automatically, and the rule keeps calling the same address. Deactivating (active=false) undeploys
the service and deactivates the rules; the versions stay, ready to be activated again.
5. The rule scores every reading
The rule a plan creates is an ordinary ADVANCED rule: it
triggers on the datastream the plan cares about, builds the request the model expects, calls the inferencer with
http.client, and translates the answer into your data model — typically a boolean datastream saying whether the
reading was anomalous, a score, an explanation, and an alarm that opens when the entity turns anomalous and
closes when it returns to normal. The rule is created inactive and is switched on and off together with the
inferencer, so a deactivated model never leaves a rule calling a service that is not there.
The rule is yours: you can read it, tune its thresholds or change what it collects in the rules editor like any other. Deleting the inferencer deletes its rules too, unless another inferencer still uses them.
Naming, in one table
Once you know the organization name and the model name, you can predict every other name the feature creates:
| Thing | Name | Example for organization acme, model radius-anomalies |
|---|---|---|
| Scheduler entry (image execution or pipeline) | <model> |
radius-anomalies |
| Exported time series file | <model>-<plan id>-<time series id>.parquet |
radius-anomalies-…-….parquet |
| MLflow experiment | <organization>-<model> |
acme-radius-anomalies |
| Registered model | <organization>-<model>-<algorithm> |
acme-radius-anomalies-isolation-forest |
| Inference image | <organization>-<model>:v<version> |
acme-radius-anomalies:v3 |
| Inferencer | <model> |
radius-anomalies |
| Deployed service | <organization>-<model> (lowercase, _ → -) |
acme-radius-anomalies |
| Inferencer endpoint | https://<service>:8443/api/predict |
https://acme-radius-anomalies:8443/api/predict |
| Rule | <model>, in default_channel |
radius-anomalies |
Retraining
A trainer with a schedule.expression runs every time the expression fires, and each run adds a new version
to the same inferencer. The scheduler understands standard five-field cron and the extended form with a leading
seconds field and a trailing year field; the web console writes seven-field expressions such as
0 0 0 15 */3 ? * — midnight on the 15th, every third month. If the expression pins a single instant (every field
numeric, year included), the trainer is a one-off and the API reports hasRetraining: false.
A trainer created without a schedule runs once, about a minute after it is created.
An inferencer keeps a bounded number of versions (five by default). When a new one arrives over the limit, the oldest inactive version is dropped; the active one is never removed by a retraining.
Callbacks and history
Because trainings take minutes to hours, nothing in this loop blocks. The scheduler records every execution in
its history (GET /scheduler/organization/{organizationId}/history), with a state — IN_PROGRESS until the
callback arrives, FINISHED when it does, FINISHED OUT OF TIME if it arrived after the wait expired — and one
entry per step with its result and description. For a time-series trainer that is two steps: the export and the
training. The web console’s See history action is a view of exactly this.
What you need
- A user with the
rootorsuper_admin_domainprofile: the five AI services accept no other. - Authentication is the usual
X-ApiKeyheader, orAuthorization: Bearer <JWT>. - The AI services are exposed on the same host as the rest of the OpenGate API, under the prefixes
/ai(training plans, trainers, inferencers),/schedulerand/fileConnector.