AIVA: from thousands of detections to a handful of situations

AIVA brings camera surveillance back from thousands of detections to a handful of well-founded situations: in 24 hours of footage from 10 cameras, 10 remained that the operator needed to see.

Camera surveillance today mostly creates work. Detection rules generate thousands of alerts, of which only a small share matters. The number of cameras grows faster than the number of operators who can watch them. And every camera needs its own zones, time windows and rules for what is normal; setting that up does not scale.

In this article we explain how AIVA works: the three analysis layers, the agents that carry out the deep analysis, the pipeline underneath, and why the costs follow what happens rather than the number of cameras.

Three analysis layers, each more expensive and smarter than the last

AIVA analyses footage in three layers. Each layer is slower and more expensive per frame, but also sees fewer frames, because the layer before it has already filtered out most of them. The heavy work only reaches frames where something is going on.

1. Traditional detection (about 10 ms per frame). Motion detection and object detection look at every frame: is something moving, is it a person or a vehicle, and where is it? This layer is fast and cheap and runs continuously on every camera.
2. Fast image analysis, "System 1" (about 150 ms per frame). Only frames with activity move on. A visual model checks them against rules that fit the situation: should this person be here at this time, is this vehicle in a place where it is not allowed?
3. Deep analysis (about 15 seconds per window). What the second layer finds suspicious is examined as a window of frames over time and across cameras. This is where the context, the pattern and the report come from.

In the pilot data of 24 hours across 10 cameras, the first layer processed 2.8 million frames. Of those, 62,000 (about 2%) went on to the fast analysis, and 420 windows to the deep analysis. Ten situations were passed on to the operator.

The deep analysis: a team of agents that follows the playbook

In the third layer, not one model is at work but a small team of agents, each with its own role. They check a situation against the customer's playbook, the way an experienced operator would.

• Orchestrator. Receives the enriched event from the second layer and decides which playbook applies, which cameras are involved and which time window needs to be examined.
• Protocol agent. Retrieves the customer's playbook and checks which approach it prescribes.
• Analysis agent. Assesses the frames, earlier passes by the same person or vehicle, and what neighbouring cameras saw.
• Verification agent. If the evidence is not conclusive, the question goes back to the orchestrator for extra frames or a different time window.
• Reporting agent. Puts the conclusion, the recommended action and the evidence frames into a report for the operator.

The operator reviews the report: confirmed or false alarm. That judgement goes back into the system and sharpens the rules, so the same false alarm is filtered out earlier next time.

The pipeline: separate services around Kafka, scalable per step

Underneath the three layers is a pipeline of separate services that talk to each other through Apache Kafka. Each step can be scaled on its own, without changing the rest.

1. Capture. Camera streams come in over RTSP. A frame publisher grabs frames at a set frame rate, stores the image in the database and puts only a lightweight reference on Kafka. No heavy images travel over the message bus, only features and references.
2. Pre-filter. The first layer (motion and object detection) reads those references, assesses the frame and adds the result. Frames without activity do not go on to the language models.
3. Batch. Apache Flink groups the frames per camera into short sequences, so a model sees a clip over time instead of loose stills. Several cameras can be combined into one frame, so the model can also connect what happens across cameras.
4. Analyse and alert. The visual model analyses the sequences, the agents turn them into a situation, and a separate service converts the result into an alert for the operator.

Scaling happens per step. The workers of a step share the work through Kafka: when cameras are added, you deploy more workers on the step that carries the heaviest load. Every step has a defined way of handling failures: a message is only marked as done once it has been processed, and a camera that drops out is picked up again. The figures in this article come from 24 hours of footage from 10 cameras that went through the full pipeline this way.

Costs follow incidents, not the number of cameras

The expensive computing power is in the language models, and they only run where something happens. As a result, costs grow with the number of incidents, not with the number of cameras.

• The first layer runs on every camera, on standard hardware and without tokens. It is the only part that grows with the number of cameras, and it is the cheapest per frame.
• The fast analysis sees about 2% of the frames. This is where the first tokens are used, for a short check of about 150 ms per frame.
• The deep analysis uses the most tokens per run, but in the pilot data it ran 420 times in 24 hours across 10 cameras: fewer than two times per camera per hour.

For a quiet site this means low, predictable costs. On a busy night usage goes up, but that is also when the system delivers the most: exactly at that moment it picks out the situations an operator would otherwise miss.

The pricing follows the same logic. Coverage, the detection and fast analysis on every camera, is a fixed amount per group of cameras. The deep analysis of incidents falls within an agreed volume, and continuous deep analysis during an ongoing incident or event is charged per hour, only when it is used.