# Image anomaly detection

## What it detects

Given a photograph of an item — a part on a line, a meter, a connector — the model says whether it looks like
the *correct* examples it was trained on or like the *incorrect* ones, and when it finds a defect it produces a
**heat map**: the same image with the suspicious region painted over, saved next to the original.

## Data it needs

This plan takes a **file source only**: a folder in your organization's [file space](../../file_connector/) with
two sub-folders of `.jpg`, `.jpeg` or `.png` images:

```text
<your folder>/
  correct/      photographs of items that are fine
  incorrect/    photographs of items with the defect
```

Upload the images — a `.zip` or `.tar.gz` is extracted on arrival — and give the trainer the folder as
`source.path`. Both classes need at least `minDataToTrain` images each; the training fails with a message naming
the class that falls short. The more varied the *correct* set, the fewer false alarms.

## How it trains

Two models are trained and used together:

- A **ResNet-18 classifier**, pre-trained on ImageNet and fine-tuned on your two folders to output the
  probability that an image is defective. Images are resized to 224 × 224 and lightly jittered in brightness and
  contrast so the model does not learn the lighting of your photo booth.
- A **PaDiM** anomaly model on the activations of one of the network's inner layers, which estimates how far each
  region of a new image is from the distribution of correct images — this is what the heat map comes from, and it
  catches defects the classifier has never seen.

The plan's metric is the classifier's **F1 score** on the test split; it is recorded with the version.

## The prediction request

The inferencer answers `POST /api/predict` with the path of an image **relative to the organization's file
space** — the same space the file connector manages, mounted for the inferencer at `/data`:

```json
{ "image_route": "/line-3/2026-09-03/part-0412.jpg", "generate_heat_map": true }
```

```json
{
  "predictions": 1,
  "anomaly_score": 0.87,
  "heatmap_path": "/line-3/2026-09-03/part-0412_heatmap.jpg"
}
```

| Field | Meaning |
|---|---|
| `predictions` | `1` defective, `0` correct |
| `anomaly_score` | Between `0` and `1`. The classifier's probability when it fires; otherwise PaDiM's distance mapped onto the same range |
| `heatmap_path` | When the item is defective and `generate_heat_map` was not `false`: the heat map written next to the original as `<name>_heatmap.<ext>`. `null` otherwise |

An `image_route` that does not exist returns `422`.

The flow is: the classifier decides first; if it sees a defect the answer is its probability and a Grad-CAM heat
map of what it looked at. If it sees nothing, PaDiM gets a second look and can still call the item defective when
its distance exceeds the plan's minimum, with its own heat map.

## The rule it creates

The rule named after the model, in `default_channel`, inactive until the inferencer is activated:

1. Triggers on the datastream **`imagePathToCheck`** — collect the path of a new photograph into it, and the rule
   runs.
2. Sends that path to the inferencer, with `generate_heat_map` taken from the rule parameter `requestHeatMap`.
3. Collects `imageWithAnomaly` (`true` / `false`) and, when there is one, `imageWithHeatMapPath`, dated at the
   photograph's timestamp.
4. Opens the alarm **`imageWithAnomaly`** (severity `URGENT`, priority `MEDIUM`) naming the image when the entity
   becomes anomalous, and closes it when a later image is correct.

So a camera integration only has to do two things: drop the photograph into the organization's file space and
collect its path into `imagePathToCheck`. The rest is the loop described in [How it works](../../how_it_works/).