Method

How we tested the estimator, in full.

The criteria were written before any measurement and never changed after reading the results. Here is everything: what passed, what failed, and what is still unmeasured.

Updated 2026-10-18

What was tested

  • The model served by the site: FairFace age outputs, compressed to 8 bits, with the same preparation as the site (face alignment, four views, averaged probabilities).
  • Test set: the FairFace validation split, under CC BY 4.0, never used for training. Labels are perceived age groups (majority of three annotators), the same kind of answer the site gives.
  • Known limit: same source (Flickr) and same annotation process as the training data. It is not an out-of-distribution test, and we cannot rule out that one person appears in both splits.
  • Two levels: the model alone on a random draw of 3,000 faces, and the full site pipeline in a browser (detection, alignment, four views) on stratified draws of 50 faces per age group, a new draw for each measurement.
  • Statistics: exact group, within one group, mean and median error in groups, 95 % Wilson confidence intervals, detection failures.

Criteria fixed before measuring

CriterionThreshold3rd measurement
Within one group, full pipeline, ages 20-69≥ 85 %94.7 %, passed
Exact group, full pipeline, adults≥ 50 %55.6 %, passed
Largest gap between population groups (adults, model alone, groups of 100+)≤ 10 points4.1 points, passed
No face found≤ 5 %7.5 %, failed
Same group when light changes (×0.6 and ×1.4)≥ 80 %83.3 %, passed
8-bit vs original model, accuracy gap≤ 1 point0.1 point, passed

Three measurements, one day

  1. First measurement: failed. The model alone was good (96.5 % within one group), but the site cropped faces differently from the training photos: 72.5 % within one group for ages 20-69 and 30.6 % exact for adults.
  2. Second measurement, crop recalibrated on the train split only: 93.8 % within one group, 44.2 % exact, and 9.1 % of faces not found. Still failed.
  3. Third measurement, alignment on 14 face landmarks, as the training photos were prepared: 5 criteria out of 6 passed. Only the “no face found” rate (7.5 %) fails.

Each change was tuned on the train split and measured on a fresh draw of validation faces never seen before.

Accuracy per age group (model alone)

True groupFacesExact groupWithin one group
0-25971.2 %96.6 %
3-939177.7 %97.2 %
10-1931446.5 %97.8 %
20-2988973.3 %98.2 %
30-3965647.7 %97.0 %
40-4935045.7 %94.6 %
50-5922645.6 %89.8 %
60-698046.2 %91.2 %
70+3537.1 %97.1 %

Exact accuracy is highest for children and 20-29 year-olds, and lowest from 30 onwards, where neighbouring groups are often confused. Groups with few faces (0-2, 60-69, 70+) have wide uncertainty: the 70+ figure rests on 35 faces only.

Accuracy per population group (model alone, adults)

Group (dataset label)Within one group
Black94.5 %
East Asian95.2 %
White95.4 %
Indian96.5 %
Southeast Asian96.6 %
Latino Hispanic97.0 %
Middle Eastern98.5 %

The largest gap is 4.1 points. Group labels come from the dataset; Face-Age never infers them about visitors, and the shipped model cannot output them.

Confusion matrix (full site pipeline, third measurement)

True \ predicted0-23-910-1920-2930-3940-4950-5960-6970+
0-230131······
3-92373······
10-19·7228·····
20-29··3314····
30-39··1112441··
40-4911·2141631·
50-59···1·10178·
60-69·····214162

310 faces detected out of 399 drawn (30 with no face found, 59 refused for several faces). Bold cells are correct answers. Most errors land in the neighbouring group; from 40 onwards, faces tend to be placed one group younger. No 70+ face was left unused in the validation split, so that group is absent here.

Light stability

On 108 faces, the image was darkened (×0.6) and brightened (×1.4). Same group in both versions: 83.3 % (95 % interval 75.2 to 89.2 %). Same group when darkened: 88.9 %; when brightened: 91.7 %.

What is still open

  • Faces not found on 7.5 % of test images, mostly in profile, turned down or partly hidden.
  • No measurement on real selfies yet.
  • The 70+ group not measured on the full pipeline.
  • A fourth measurement on an independent dataset (LAGENDA, different source, apparent-age votes) is written in the protocol, with the same criteria; its results will be published here, whatever they are.

On 18 October 2026, the publisher decided to remove the PROTOTYPE badge with 5 criteria out of 6 met. See method and limits.