How we tested the estimator, in full.
The criteria were written before any measurement and never changed after reading the results. Here is everything: what passed, what failed, and what is still unmeasured.
Updated 2026-10-18
What was tested
- The model served by the site: FairFace age outputs, compressed to 8 bits, with the same preparation as the site (face alignment, four views, averaged probabilities).
- Test set: the FairFace validation split, under CC BY 4.0, never used for training. Labels are perceived age groups (majority of three annotators), the same kind of answer the site gives.
- Known limit: same source (Flickr) and same annotation process as the training data. It is not an out-of-distribution test, and we cannot rule out that one person appears in both splits.
- Two levels: the model alone on a random draw of 3,000 faces, and the full site pipeline in a browser (detection, alignment, four views) on stratified draws of 50 faces per age group, a new draw for each measurement.
- Statistics: exact group, within one group, mean and median error in groups, 95 % Wilson confidence intervals, detection failures.
Criteria fixed before measuring
| Criterion | Threshold | 3rd measurement |
|---|---|---|
| Within one group, full pipeline, ages 20-69 | ≥ 85 % | 94.7 %, passed |
| Exact group, full pipeline, adults | ≥ 50 % | 55.6 %, passed |
| Largest gap between population groups (adults, model alone, groups of 100+) | ≤ 10 points | 4.1 points, passed |
| No face found | ≤ 5 % | 7.5 %, failed |
| Same group when light changes (×0.6 and ×1.4) | ≥ 80 % | 83.3 %, passed |
| 8-bit vs original model, accuracy gap | ≤ 1 point | 0.1 point, passed |
Three measurements, one day
- First measurement: failed. The model alone was good (96.5 % within one group), but the site cropped faces differently from the training photos: 72.5 % within one group for ages 20-69 and 30.6 % exact for adults.
- Second measurement, crop recalibrated on the train split only: 93.8 % within one group, 44.2 % exact, and 9.1 % of faces not found. Still failed.
- Third measurement, alignment on 14 face landmarks, as the training photos were prepared: 5 criteria out of 6 passed. Only the “no face found” rate (7.5 %) fails.
Each change was tuned on the train split and measured on a fresh draw of validation faces never seen before.
Accuracy per age group (model alone)
| True group | Faces | Exact group | Within one group |
|---|---|---|---|
| 0-2 | 59 | 71.2 % | 96.6 % |
| 3-9 | 391 | 77.7 % | 97.2 % |
| 10-19 | 314 | 46.5 % | 97.8 % |
| 20-29 | 889 | 73.3 % | 98.2 % |
| 30-39 | 656 | 47.7 % | 97.0 % |
| 40-49 | 350 | 45.7 % | 94.6 % |
| 50-59 | 226 | 45.6 % | 89.8 % |
| 60-69 | 80 | 46.2 % | 91.2 % |
| 70+ | 35 | 37.1 % | 97.1 % |
Exact accuracy is highest for children and 20-29 year-olds, and lowest from 30 onwards, where neighbouring groups are often confused. Groups with few faces (0-2, 60-69, 70+) have wide uncertainty: the 70+ figure rests on 35 faces only.
Accuracy per population group (model alone, adults)
| Group (dataset label) | Within one group |
|---|---|
| Black | 94.5 % |
| East Asian | 95.2 % |
| White | 95.4 % |
| Indian | 96.5 % |
| Southeast Asian | 96.6 % |
| Latino Hispanic | 97.0 % |
| Middle Eastern | 98.5 % |
The largest gap is 4.1 points. Group labels come from the dataset; Face-Age never infers them about visitors, and the shipped model cannot output them.
Confusion matrix (full site pipeline, third measurement)
| True \ predicted | 0-2 | 3-9 | 10-19 | 20-29 | 30-39 | 40-49 | 50-59 | 60-69 | 70+ |
|---|---|---|---|---|---|---|---|---|---|
| 0-2 | 30 | 13 | 1 | · | · | · | · | · | · |
| 3-9 | 2 | 37 | 3 | · | · | · | · | · | · |
| 10-19 | · | 7 | 22 | 8 | · | · | · | · | · |
| 20-29 | · | · | 3 | 31 | 4 | · | · | · | · |
| 30-39 | · | · | 1 | 11 | 24 | 4 | 1 | · | · |
| 40-49 | 1 | 1 | · | 2 | 14 | 16 | 3 | 1 | · |
| 50-59 | · | · | · | 1 | · | 10 | 17 | 8 | · |
| 60-69 | · | · | · | · | · | 2 | 14 | 16 | 2 |
310 faces detected out of 399 drawn (30 with no face found, 59 refused for several faces). Bold cells are correct answers. Most errors land in the neighbouring group; from 40 onwards, faces tend to be placed one group younger. No 70+ face was left unused in the validation split, so that group is absent here.
Light stability
On 108 faces, the image was darkened (×0.6) and brightened (×1.4). Same group in both versions: 83.3 % (95 % interval 75.2 to 89.2 %). Same group when darkened: 88.9 %; when brightened: 91.7 %.
What is still open
- Faces not found on 7.5 % of test images, mostly in profile, turned down or partly hidden.
- No measurement on real selfies yet.
- The 70+ group not measured on the full pipeline.
- A fourth measurement on an independent dataset (LAGENDA, different source, apparent-age votes) is written in the protocol, with the same criteria; its results will be published here, whatever they are.
On 18 October 2026, the publisher decided to remove the PROTOTYPE badge with 5 criteria out of 6 met. See method and limits.