Home/Work/Real-time sports analytics engine

Case study · Streamingo AI · Sports media and analytics · 2022–24

Real-time sports analytics engine

A live video pipeline combining object detection, multi-object tracking, pose estimation and court localisation, built on Swin, VideoMAE and ViT backbones, then optimised with TensorRT and ONNX to deliver 3x the frame rate for real-time analysis.

3xFrames per second, identical hardware
4Vision stages at full frame rate
2Precisions, INT8 and FP16
3Backbones: Swin, VideoMAE, ViT

§ 01 — Problem

Frame-accurate sports analytics needs detection, multi-object tracking, pose estimation and court localisation running together at full frame rate, on a fixed GPU budget. Each stage is a heavy vision model, and all four share the same GPU. Run one after another at native precision, the pipeline falls behind the video and analysis becomes an offline job.

The requirement was a single pipeline that keeps every stage in step with the incoming frames, across several streams at once, on the hardware already in place.

§ 02 — Architecture

Four stages, one GPU.

Detection, tracking, pose and court localisation run as TensorRT engines on one GPU, batched across streams.

Fig. 1System architecture
Decoded frames are batched across streams and run through four TensorRT engines: detection on a Swin backbone, multi-object tracking, pose estimation on a ViT backbone and court localisation by homography. A VideoMAE model reads the sequence for events. Every stage was exported through ONNX and compiled with graph fusion and INT8/FP16 quantisation.

§ 03 — How it works

One frame, four stages.

The animation follows live frames through decode, detection, tracking, pose, court localisation and events, with the per-stage latency and the frame-rate gain.

Fig. 2Pipeline trace (live illustration)
Player positions, IDs, latencies and confidences in the trace are sample values, not client data.
FilmCase film, 23 seconds
1280 × 720 · H.264 · no audioDownload MP4

Diagrams and films on this page are illustrations of the system's architecture and behaviour, drawn from my work at Streamingo AI. They are not screenshots or recordings of production systems, which remain confidential.

§ 04 — Approach

Models

  • Object detection and multi-object tracking for players and ball, with identities held across frames.
  • Pose estimation for each tracked player, and court localisation to place every position on the court.
  • Swin, VideoMAE and ViT backbones across the four stages.
  • Related work in the same role: temporal event-understanding systems with Swin, VideoMAE, MVD, ViT and BEiT; zero-shot and few-shot classification with CLIP, Detic and Prototypical Networks for rapid scaling to unseen classes.

Inference

  • Every stage exported through ONNX and compiled into a TensorRT engine.
  • Graph fusion inside each engine.
  • INT8/FP16 quantisation, chosen per engine.
  • Batched inference across streams, so several feeds share one GPU at full frame rate.

§ 05 — Impact

3x the frames per second on identical hardware, which moved analysis from offline processing to true real-time.

What generalises: the frame-rate gain came from the whole chain, so every stage was compiled; precision was set per engine between INT8 and FP16; and batching across streams filled the GPU time that a single stream left idle.

Object detectionMulti-object trackingPose estimationCourt localisationSwinVideoMAEViTTensorRTONNXINT8/FP16Batched inference

§ 06 — At a glance

A one-page summary of this case for sharing. PNG, 1080 × 1350.

Download PNG
At-a-glance infographic for the real-time sports analytics engine