Scheduler
Limited access
What the scheduler does
The scheduler runs things later, and again. Three kinds of things:
| Type | What runs | Typical use |
|---|---|---|
| REST request | An HTTP call to any URL, with headers and body | Start a time series export every night |
| Image execution | A container image from the platform registry, as a Kubernetes Job | Run a training plan, an aggregation, a report |
| Pipeline | Two or more of the above, in sequence | Export data, then train on it — which is exactly what a time-series trainer is |
Every schedulation belongs to an organization, has an identifier you choose, a schedule and a history of executions. Trainers are its main customer today, but the console’s Schedulers screen and the dashboard scheduler widgets let you use it directly.
| To | Call |
|---|---|
| Create | POST /scheduler/organization/{organizationId}/restRequest · …/imageExecution · …/pipeline |
| List | GET on the same three paths |
| Delete | DELETE …/restRequest/{requestId} · …/imageExecution/{imageExecutionId} · …/pipeline/{pipelineId} |
| Report completion (callback) | POST …/{type}/{id}/execution/{executionId} — for a pipeline, …/{executionId}/{stepId} |
| Cancel a running execution | DELETE …/imageExecution/{id}/execution/{executionId} · …/pipeline/{id}/execution/{executionId}/{stepId} |
| Read the history | GET /scheduler/organization/{organizationId}/history |
The schedule
Every schedulation carries the same schedule object, in one of two forms:
| Field | Meaning |
|---|---|
cron.expression |
Standard five fields (minute hour day month weekday), optionally with a leading seconds field and a trailing year field. 0 0 2 * * * * is every day at 02:00:00. ? is accepted in the day fields |
cron.timeZone |
Where the expression is evaluated. Default UTC |
interval.minutes |
Run every n minutes instead, counted from from or from creation |
executeNow |
Also run immediately on creation |
from |
Cron: no execution before this instant. Interval: the first execution |
to |
No execution after this instant |
If an execution is still running when the next one is due, the next one is skipped and logged, not queued.
An invalid time zone or expression is rejected on creation with 400 and the offending field named in the error.
REST requests
response says how the scheduler knows the request is done:
| Form | Behaviour |
|---|---|
"sync": { "timeout": 5 } |
The request is complete when the HTTP response arrives. timeout is the seconds to wait for it. A 4xx/5xx marks the execution as an error, with the platform error message when the body carries one |
"async": { "maxTimeToWaitCallback": 600 } |
The scheduler adds a callback header to the outgoing request holding the URL the target must POST to when its work is done, and waits up to this many seconds for it. The platform’s own asynchronous endpoints, such as the time series export, honour that header |
Image executions
| Field | Meaning |
|---|---|
imageExecution.name, tag |
The image, resolved in the platform’s registry |
env |
Environment variables for the container |
envFrom |
Kubernetes secrets and config maps to expose as environment, optionally with a key prefix |
timeout |
Seconds the job may run before Kubernetes kills it |
maxTimeToWaitCallback |
Seconds to wait for the container to report completion, normally a little more than timeout |
The container runs as a Kubernetes Job with no retries, with the organization’s file space
mounted at /data, and with one extra environment variable the scheduler adds itself: callbackUri, the URL
the container must POST to when it finishes. An image that cannot be pulled, or a container that exits with an
error before reporting, fails the execution with that reason in the history.
Pipelines
A pipeline is a list of at least two steps, each a REST request or an image execution with a step
identifier unique in the pipeline. Steps run in order: a synchronous REST step hands over as soon as its
response arrives; an asynchronous REST step and an image step hand over when their callback arrives at
…/pipeline/{pipelineId}/execution/{executionId}/{stepId}. A step that fails stops the pipeline; the history
records the failing step and its description, and the remaining steps are not run.
The example above is, field for field, what the Trainers API creates for a time-series trainer.
Callbacks
Asynchronous work reports back with a POST to the callback URL — the one in the callback header for a REST
request, in callbackUri for a container — carrying:
result is free text by contract; the platform’s own jobs use OK, ERROR and TIMEOUT. The callback is
authenticated like any other call to the scheduler. It answers 204.
A callback that arrives after maxTimeToWaitCallback is not lost: the execution, already marked as finished
without a callback, moves to FINISHED OUT OF TIME and records the late result.
Execution history
| Filter | Meaning |
|---|---|
schedulerType |
REST_REQUEST, IMAGE_EXECUTION or PIPELINE |
schedulerId |
The schedulation identifier — for a trainer, the model name |
limit |
Maximum number of entries |
state is IN_PROGRESS while a callback is awaited, FINISHED when the execution completed — look at each
step’s result to know how — and FINISHED OUT OF TIME when the callback arrived after the wait had expired.
An execution stopped by hand shows a step with result CANCELLED.
Cancelling a running execution
Deletes the Kubernetes Job behind an image execution — or the image step of a pipeline — waits until it is gone,
and records the step as CANCELLED in the history. The schedulation itself is untouched and fires again at the
next tick; to stop that, DELETE the schedulation.
In the web console
Everything on this page has a dashboard widget: the Image Execution, Rest Request and Pipeline scheduler browsers, their wizards, and the Schedulers History list. The Artificial Intelligence section of the console has its own, simpler Schedulers screen — see The web console.