What to decide before ordering detection AI, and why performance is hard to promise

There are four things to decide before ordering. What to find, where the images are taken and where the model runs, who uses the results and how, and how many misses and false detections are acceptable. How training photos are collected and how the model is evaluated both follow from these four. The type of model and the number of photos come after.

How would you decide them for road damage?

What to decide	Example for road damage	What happens if you don't
Target	Only potholes, or cracks too? From what size?	The rules for annotation (drawing ground-truth boxes) vary from person to person
Setting	Shot on a smartphone, a dashcam or a dedicated vehicle? Where does it run?	Training photos and field photos don't match
Use	Does staff check each candidate one by one, or are they totted up as counts?	It's unclear which evaluation numbers to look at
Acceptable errors	Reduce misses, or reduce false detections?	There's no standard for calling it finished

Put the target into words, including size and severity, not just its name. Whether you want to know where holes are, or to diagnose the degree of pavement deterioration section by section, changes what gets built. The crack ratio (%) used in pavement diagnosis, for example, can't be produced by a detection model that just draws boxes on photos (for the indicators, see the summary of Japan's Pavement Inspection Guidelines).

Misses or false detections: which should you reduce?

A road seen from above with real damage, a hole and two cracks, plus a shadow and a manhole. The detection AI correctly boxes the hole and one crack, misses the other crack, and wrongly boxes the shadow — Fig. 1 A miss is real damage without a box; a false detection is a box on something that isn't damage

A miss is failing to detect damage that's really there. A false detection is detecting something that isn't damage, such as a shadow or a stain. In evaluation, each is expressed with these metrics (Source: Ultralytics Docs).

Metric	Share of what	When high
Precision	Results reported as "found" that were correct	Few false detections
Recall	Real objects that were detected	Few misses

Detection AI gives each candidate a score (confidence) and keeps only those above a threshold. Raising the threshold reduces false detections, but if a discarded candidate was real damage, it becomes a miss.

Which to reduce depends on the use. If staff check every candidate by eye, you can lean towards fewer misses even at the cost of more false detections. If results are used as counts without anyone checking, both misses and false detections throw the numbers off. Potholes are damage that Japan's Pavement Inspection Guidelines say should be "dealt with as soon as found", so if they are a target, weigh misses heavily (Source: Pavement Inspection Guidelines, p. 3).

How many training photos do you need?

The number needed depends on the target and the shooting conditions, so it can't be fixed in advance, because performance on photos taken under different conditions from training isn't guaranteed. METI's contract guidelines (AI section) also say that model performance depends on the training data and behaviour on unseen inputs is unclear, so guaranteeing performance at the time of contract is difficult (Source: METI).

So first list the range of photos that will be taken in the field, try it small, and add photos for the conditions that are missing.

Camera equipment and mounting (smartphone, dashcam, dedicated vehicle)
Driving speed, time of day, weather, season
Road condition (new pavement, old pavement, repair patches)
Things that look like damage (shadows, stains, puddles, white lines, manholes)

You can also try the public RDD2022 dataset (47,420 road images from six countries including Japan, covering three types of crack and potholes), but in the end confirm on photos from your own site (Source: Arya et al., arXiv:2209.08538).

How do you decide the evaluation?

Evaluate on photos not used in training. Split off the evaluation photos before training starts, and take them on different days and routes from the training photos. Mixing frames cut from the same video into both makes evaluation numbers look better than they are. The ordering side, who know the site best, should take the lead in choosing evaluation photos.

Match the counting to the units of the work, too: decide in advance whether to count by damage size, and how many metres off a position can be and still count as the same damage. MLIT's catalogue of inspection support technologies, for example, splits potholes into under 10 cm, 10 to 20 cm and 20 cm or more, and judges whether each was shown within 5 m of its true position (Source: MLIT).

The mAP figure often seen in papers gives a rough sense of performance. What the work needs to know is how many misses and false detections there were at the chosen threshold. Ask for photos of missed examples and false detections to be included in the report.

How should the contract be split?

METI's guidelines describe an "exploratory, staged" development approach in four stages: assessment, PoC (proof of concept with a prototype), development, and additional training. Since you decide whether to continue after seeing each stage's results, you can stop as soon as it's clear performance won't be good enough.

Flowchart of detection AI development in four stages: assessment (deliverable: report), PoC (report and prototype model), development (trained model), additional training (retrained model). After assessment and after PoC, you decide whether to continue — Fig. 2 The four stages and their main deliverables. Decide whether to continue after the first two

Dividing cost and time by stage also makes the burden easier to foresee for the ordering side. What most affects the price is the variety and difficulty of the targets, the range of shooting conditions and the amount of annotation.

Also decide before ordering whether additional training after delivery goes into the contract. Once in use, photos from conditions no one anticipated will turn up. Discuss who will review the model, how often, and with which photos. Issues of contract form and rights to deliverables can be checked in METI's "Contract checklist for using and developing AI" (Source: METI).

Can YOLO be used commercially?

You can't say across the board that "YOLO can be used commercially" or "can't". Detection models named YOLO come from more than one provider, and the terms differ by provider. When building on a published model, check the terms of the code, the trained weights and the training data separately. These are the terms shown on the providers' pages as checked on September 27, 2026.

Name	Code	Weights	Training data
Ultralytics YOLO	AGPL-3.0, or a paid Enterprise License	Models you train are also covered by AGPL-3.0 by default	No statement found
YOLOX	Apache License 2.0	No statement found	No statement found
RF-DETR	Core under Apache License 2.0; Plus components under PML 1.0	Mostly Apache License 2.0; detection XL and 2XL under PML 1.0	No statement found
RDD2022 (dataset)	LICENSE file is MIT	Not applicable	Images: CC BY-SA 4.0 in the README, CC BY 4.0 on figshare; the wording differs by source

(Sources: Ultralytics License, YOLOX LICENSE, rf-detr, RoadDamageDetector, figshare)

According to Ultralytics, using YOLO's code or models trained with it requires either publishing your whole project under AGPL-3.0 or obtaining an Enterprise License. This applies even if you train from scratch or use it only internally.

When ordering, ask for a list of the code, weights and data used, with names, versions, licenses and the date checked. Also ask, before signing, who is responsible for meeting which terms.

What to prepare for a consultation

You don't need perfect documents. If the following are known, things move faster.

What you want found, with example photos (clear ones, borderline ones, and look-alikes that aren't targets)
Camera equipment, how photos are taken, and where it will run (smartphone, field device, cloud)
Who checks the results, how, and what they're used for
Photos you can set aside for evaluation, and people who can judge the ground truth
Rights to the photos, how faces and number plates in shot are handled, and whether the data you hand over may be used for anything besides development

The information on contracts and licenses is not legal advice. Check the original text before use, and if in doubt, ask the provider or a professional such as a lawyer.

If you'd like to discuss building detection AI Infra Intelligence builds apps with AI models built in, matched to what you need to find. From sorting out requirements to training, evaluation and retraining in operation, the work starts small and grows from there.

Last checked: September 27, 2026

What to decide before ordering image detection AI, and why performance is hard to guarantee in a contract