Detecting curve mirrors with AI in images taken while driving
A detection model that finds roadside curve mirrors in images of the road ahead taken by a smartphone in a car. Mirrors on their own posts and mirrors on utility poles are boxed differently.
Model Lab has released a detection model that finds roadside curve mirrors in images of the road ahead taken by a smartphone in a car. A mirror on its own orange post is detected with a tall box covering both mirror and post; a mirror fixed to a utility pole or similar is detected by the mirror face alone. The weights are published for developers under Apache-2.0 and can be built into business systems.
What are curve mirrors for?
They show cars and people coming from behind buildings or walls at blind bends and junctions. You often see them on the corners of narrow residential streets and on turns halfway up a hill. Most are a round mirror on top of an orange post, but some are fixed to utility poles or walls.
Mirrors get knocked out of alignment, get too dirty to show anything, or lean on their posts. To notice that, you first need to know where the mirrors are.
Why build this model?
When planning inspections or rebuilding an inventory of road furniture, road managers often have to confirm each mirror's position one at a time. Searching by eye on site takes longer the more narrow streets an area has.
If mirrors can be pulled out of images taken while driving, a single car can do the screening before the site visits. For that, a model with the same architecture as YOLOX-Tiny was trained from scratch, without pre-trained weights.
Who is it for?
- Local government staff in charge of curve mirror inspection or road furniture inventories
- Companies contracted for inspections or inventories
- Companies and researchers who build maps and road data from driving footage
How does it find them?
There is one target class, curve mirrors; only the box changes depending on whether there is a post. Things that look like mirrors, such as round signboards, were learned as a separate class so the model can tell them apart. The model returns scores for two classes: curve mirror and mirror look-alike.
Each image is shrunk so its long side is 1280 pixels. A 16:9 image is then 720 pixels tall, so the bottom 16 pixels are filled with grey to make 1280 × 736. For each image the model returns 200 candidates with a position and a score (confidence, 0 to 1).
Duplicate candidates are merged, and those with a curve mirror score of 0.66 or higher count as detections. The 0.66 was set on validation images so that there are no more than three false detections per 100 images without mirrors. Merging duplicates (NMS) is left to the developer building the model in.
How well does it find them?
Counted on 3,997 test images (465 mirrors) taken in areas not used in training. The threshold was fixed before looking at the test images.
- Recall (share of mirrors found): 0.641
- Precision (share of detections that were correct): 0.661
- False detections per 100 images with no mirrors: 2.74
By box type, recall was 0.710 for mirrors detected with their post and 0.526 for mirrors detected by the face alone. For mirrors that appear large and close it falls further, to 0.463. Some of the counted false detections may be real mirrors that were missed during annotation (drawing the ground-truth boxes); this is being checked.
What can it be used for?
It suits feeding in images collected while driving and pulling out the frames that show a mirror. Combined with where each image was taken, that gives a draft inventory of which corners have mirrors. Test recall is 0.641, so mirrors it misses need to be filled in by eye.
License and formats
The weights and code are released under Apache-2.0. They can be built into business systems and products, and modified and redistributed. Three formats are available: PyTorch, ONNX and CoreML; with CoreML it runs on an iPhone.
Things to keep in mind
The distributed ONNX and CoreML files have a fixed input of 1280 × 736. A 4:3 image has to be shrunk further to fit, and on validation images recall fell from 0.658 to 0.549. Landscape smartphone video (16:9) is not affected.
Round signboards, round streetlight shades and the backs of signs can be mistaken for mirrors. Mirrors without their own post in view, such as those on utility poles, are missed more easily. Do not make inspection or safety decisions on this model's results alone.
Model details (inputs and outputs, evaluation results, video of detections on real footage): Curve Mirror YOLOX-Tiny
Weights: Hugging Face
Apps that use models like this to find specific markers or equipment in video can also be built on request: AI App Development
Cover and figure photos: Kamikou, 24kawa / Mapillary (CC BY-SA 4.0)