It recognises me: how a face becomes a fingerprint

The first real skill I wanted was the simplest to say: when I sit down, the robot knows it is me.

Inside the machine

Recognition is two steps, done by two small open-source models from OpenCV.

1. Find the face. The first model, YuNet, looks at the camera image and draws a box around each face.
2. Turn the face into a fingerprint. The second model, SFace, takes what is inside the box and turns it into a list of 128 numbers. Two photos of the same person give two lists that point roughly in the same direction. Two different people give lists that point elsewhere.

To compare two lists, you measure the angle between them. The result, called a cosine, goes from 0, nothing in common, to 1, identical. OpenCV suggests a threshold of 0.363: above it, same person.

So the robot does not store my photo to compare pixels. It keeps a few fingerprints of my face, and compares each new one to them.

At this stage the models ran on my Mac, fed by the robot’s camera. Moving them onto the robot came later.

Enrolling: teaching it my face

To enrol, the program watches me for a minute and keeps the good images.

The first attempt failed completely: it was night, and the room was dark. Average brightness 6 out of 255, zero face found. The program now says “image too dark” at the first view instead of failing in silence.

With the light on, it kept 14 images out of 32 views. Then a check I like: it compares the kept images with each other. Ten resembled each other well. Four resembled less than half of the others, so I removed them. My face is now ten fingerprints.

What the numbers said

A test at my desk, sitting about one metre away, in office light:

Measure	Result
Images with a face	184
Verdicts “it’s me”	180
The four others	all while I was leaving the frame
Median score	0.556, for a threshold of 0.363
Time from image to verdict	20 ms median, on the Mac

What surprised me

Then I stood up, one and a half to two metres away. The median score fell to 0.28, under the threshold. The robot no longer knew me.

My first guess was size: a smaller face, fewer pixels. Wrong. I shrank my enrolment images to the same small size and they still scored 0.75. Sitting and standing at the same distance gave 0.42 against 0.34. The difference is the point of view: standing, the camera sees my face from below, and my fingerprints were all taken from the front.

There is a second lesson in that. The camera is in the robot’s head. When the head turns, my face moves in the image, but its angle does not change. To teach it other angles, I am the one who has to turn.

Next

Recognition worked at the desk. Now the robot had to find me when I was not in front of it, and do it without the Mac.