Where can't a snowplow throw snow? Finding small roadside signs with AI

On the shoulders of Japanese expressways stand small signs that mark the start and end of sections where snowplows must not throw snow. The start sign is a blue square inside a yellow square; the end sign is a yellow and blue cross. The colours are the same and only the shape differs. Model Lab has released a detection model that finds these signs in video from a smartphone in the vehicle.

The start and end signs. The start is a blue square inside a yellow square; the end is a yellow and blue cross — Fig. 1 Start sign (left) and end sign (right)

What are these signs for?

Snowplows clear the lanes by throwing snow off the side of the road. Where another road or structure passes underneath, that snow would land on the cars or structures below. A start sign stands before each such section and an end sign where it ends.

The signs sit on red and white posts, and up close their shape is easy to read. The hard part is not missing a single small, distant sign while driving a snowplow. Today the operator watches ahead and decides by eye: stop throwing here, and start again here.

Why build this model?

Road work is full of moments where you have to notice a set marker while driving and change what you do. These signs are among the easiest to miss. From a distance they are a tiny dot on screen, and start and end can only be told apart by shape.

Could one smartphone avoid both missing them and mixing them up? To find out, the model was trained from scratch on video I shot driving expressways myself, without pre-trained weights. The finished model is published with instructions and evaluation results.

Expressway driving footage. Inside the cropped area ahead on the road, the start and end signs are boxed — Fig. 2 A start sign and the end sign beyond it, detected. The dotted line is the far-field crop

How does it find them?

Each frame is processed two ways. First the video is resized to 4K (3840 × 2160), and the area ahead on the road where signs appear small is cut out at 1280 × 736 without shrinking. This far-field crop lets small signs go into the model without being squashed.

As the vehicle gets close and a sign leaves the crop, detection switches to the whole frame shrunk to a third. The model is the same; only the input image changes. The three frames below cover 2.2 seconds approaching the same end sign.

Three scenes approaching the same end sign. At 0.0 s it is detected in the far-field crop, at 1.6 s in both the crop and the full frame, at 2.2 s in the shrunk full frame — Fig. 3 Far away, the far-field crop does the work; up close, the shrunk full frame does

For each image the model returns 200 candidates with a position and a score (confidence, 0 to 1). Keeping only those above a threshold and merging duplicates (NMS) are left to the developer building the model in. The evaluation used a threshold of 0.3. Lower it to miss fewer signs; raise it for fewer false detections.

How well does it find them?

Counted on 57 test images not used in training.

Start signs: 12 of 18 detected
End signs: 17 of 21 detected
Start and end mixed up: 0
False detections (something else taken for a sign): 4

Only signs within the far-field crop were counted. The test images have been used for checks many times, so the plan is to measure again on newly shot images. Night, rain and snow have not been checked yet.

The published model was also run as-is on separate driving footage used for neither training nor evaluation. It detected a start sign at 0.85 and an end sign at 0.87, with no mix-ups. In one scene, though, it mistook a yellow line painted on a wall for a start sign. That error came from the shrunk full frame, with a fairly low score of 0.44.

Three examples: a start sign detected at 0.85, an end sign at 0.87, and a yellow line on a wall falsely detected as a start sign — Fig. 4 Successful detections and a false detection

What can it be used for?

The most obvious use is alerts on a smartphone in the snowplow cab. A prototype app with the model built in turns the screen red and sounds a buzzer when it finds a start sign, and turns green with a chime once past the end sign. So the operator never has to touch the screen while driving, it communicates only through colours you can read at a glance and sounds you can tell apart.

Each detected location is saved on the device with a photo and can be reviewed on a map. After the run it serves as a list of sign locations, a draft inventory. A sign is confirmed only after it has been seen several times on approach, so a momentary false detection does not set off the alarm.

Four app screen mock-ups: checking (grey), no throwing (red), throwing allowed (green), and a map of the run — Fig. 5 Screens with the model built into an app (based on the prototype app; map from GSI tiles)

The model weights are published for developers in three formats, PyTorch, ONNX and CoreML; with CoreML it runs on an iPhone. The license is Apache-2.0, so it can be built into business systems and products.

Things to keep in mind

It only finds these snow-throwing signs. Almost no night footage was used in training. Do not decide whether snow may be thrown on this model's results alone. It does not replace the operator's eyes; it adds one more chance to notice a sign that might be missed.

Model details (inputs and outputs, evaluation results, video of detections on real driving footage): No-Snow-Throwing Sign YOLOX-Tiny

Weights: Hugging Face

Apps that use models like this to find specific markers or equipment in video can also be built on request: AI App Development