Ultrasound Anatomy Detection

Detecting anatomical structures in ultrasound images with transformer-based detection models, which raised accuracy from 60% to 90%. Shown with a pipeline that produces labelled training images from simulation.

The problem

Ultrasound is hard to read and expensive to label. Every annotated image needs a clinician’s time, and a detector needs a great many of them.

The approach

Let the simulation do the labelling. A simulated ultrasound image is rendered from a 3D model, so the position of every structure in it is already known.

1. 3D modelling. The anatomy starts as a 3D model.
2. Ray tracing. An ultrasound image is rendered from it.
3. Automatic annotation. The labels for each structure are produced together with the image.
4. AI on synthetic data. Detection models are trained on the generated set.
5. Inference on real data. The trained model is applied to real scans.

Result

With transformer-based detection models, anatomical structure detection in ultrasound images improved from 60% to 90% accuracy.

This is one use of the scalable data-processing pipeline I designed at VirtaMed, which converts 3D simulation output into training-ready datasets for machine-learning and LLM-based models.