Detecting road signs with AI and sorting them into five types

Model Lab has released a detection model that finds road signs in images of the road ahead taken by a smartphone or dashcam in a car. It sorts each sign by shape and colour into five types: guide, warning, regulatory, instruction and supplementary. The weights are published for developers under Apache-2.0 and can be built into business systems.

Two regulatory signs on a post in front of a stone wall, each outlined in a blue box with scores of 0.93 and 0.89 — Photo: pika_sky / Mapillary (CC BY-SA 4.0)

Why find road signs?

Road signs fade, get knocked out of alignment, or end up hidden behind branches. To notice that, a road manager first needs to know which sign stands where.

When inspecting signs or rebuilding a sign inventory, staff often have to check the position and type of each sign on site, one by one. If signs can be pulled out of images taken while driving, a single car can do the screening before the site visits. Lining up the photo and location of each detected sign gives a draft inventory, or something to compare against the last record.

Who is it for?

Staff in charge of sign inspection or inventories at national, local and expressway road managers
Companies contracted for inspections or inventories
Companies and researchers who build maps and road data from driving footage

In each case it suits work where people currently pick out the frames and positions of signs from large numbers of images by eye.

How does it find them?

The model detects the sign itself, not the post. If one post carries a main sign and a supplementary sign, each gets its own box. Things that look like signs, such as shop signs, traffic lights and the backs of signs, were learned as separate classes and are removed from the output.

Four scenes: a crosswalk instruction sign and a speed limit sign; a warning sign and a regulatory sign; a left-turn instruction sign with a supplementary sign below; and three regulatory signs and one supplementary sign on one post. Each is outlined in a blue box with its class name — Photos: 24kawa, okadatsuneo / Mapillary (CC BY-SA 4.0)

Each image is shrunk so its long side is 1280 pixels; a 16:9 image then has its bottom 16 pixels filled with grey to make 1280 × 736 before it goes into the model. For each image the model returns 200 candidates with a position and a score per class (confidence, 0 to 1). Duplicate candidates are merged per class (NMS), and those scoring 0.5 or higher are treated as signs.

The model has the same architecture as YOLOX-Tiny and was trained from scratch, without pre-trained weights. It does not tell individual sign types apart (stop, speed limit and so on).

Three frames approaching the same two regulatory signs on a post. Every sign, from small and far to large and close, has a box — Photos: 24kawa / Mapillary (CC BY-SA 4.0)

How well does it find them?

Counted on 596 test images (988 signs) taken in seven regions not used in training. The threshold was fixed before looking at the test images.

Recall (share of signs found): 0.857
Precision (share of detections that were correct): 0.877
Classification accuracy (share of detected signs given the right type): 0.980
False detections per 100 images with no signs: 12.0

Bar chart of recall by class: regulatory 0.923, instruction 0.875, supplementary 0.698, guide 0.661

Regulatory and instruction signs are found well; guide signs (including intersection and street name signs) and supplementary signs are missed more often. Warning signs are not listed because the test set has only nine of them. The ground-truth boxes for the test set were drawn by AI and partly checked by the developer, not reviewed one by one.

License and formats

The weights and code are released under Apache-2.0. They can be built into business systems and products, and modified and redistributed. Two formats are available: PyTorch and ONNX.

Things to keep in mind

The distributed ONNX file has a fixed input of 1280 × 736. A 4:3 image has to be shrunk further to fit, and on validation images recall fell from 0.869 to 0.740. Landscape smartphone video (16:9) is not affected.

Signs that appear large and close are missed more easily. Shop signs, facility signboards and address plates can be mistaken for road signs. Do not make inspection or safety decisions on this model's results alone.

Model details (inputs and outputs, evaluation results, video of detections on real footage): Road Sign YOLOX-Tiny

Weights: Hugging Face

Apps that use models like this to find specific equipment in video can also be built on request: AI App Development

Cover photo: pika_sky / Mapillary (CC BY-SA 4.0)