It recognises me from the side, and now it doubts me

Since 23 October, the robot followed my face from every angle, but it only tried to recognise me when I looked more or less at it. In profile, it did not even try. I had decided it could not do it. I was wrong, and it took me three weeks to check.

Why I believed it

When it first recognised me, its score dropped as soon as I stood up: the camera saw my face from below, and my saved face was ten photos taken from the front. From there I drew a general rule: the more turned the face, the less it works.

So when the identity started following the person, I wrote a cautious rule. The program measures how turned a face is: about 0.03 when I look straight at the robot, above 1 in full profile. Past 0.5, no recognition: the robot kept the name it had given me earlier, and if it had none, it waited for me to turn back.

It was reasonable. It was also never measured.

The measurement

On 12 November I finally asked the robot. A small tool records every face its camera sees for a minute, with its angle and its score against my saved face. I sat at my desk and slowly turned my head, from three-quarter to full profile, and back.

78 faces in one minute, 78 recognised. Even in full profile, the lowest score was 0.39, above the threshold of 0.363 suggested by OpenCV for its SFace model. That model was designed to stay robust when the face is not perfect; I had simply never let it show me.

So the robot did not fail in profile. It was not allowed to try.

What changed

Two things, the same day.

1. It remembers me for longer. When a face leaves the image, the robot now keeps its track for three minutes instead of twenty seconds. If I turn away to grab a coffee, I am still me when I come back.
2. It recognises at every angle. The fingerprint of the face is now computed whatever the angle. To give a name to a face seen in profile, the bar is higher: 0.45 instead of 0.363. Installed at 1:19 pm, the robot named me 7 seconds after restarting.

That night, the higher bar bit me. Sitting two metres away in profile, in the dark, I scored 0.43 or 0.44, just under 0.45, and the robot took my name away every fifteen seconds. Since then, the higher bar is only used to give a name or change it, never to take one back.

What the numbers said next

I was ready to write “solved”. Then I read its log for Saturday 14 November.

	10 November, before	14 November, after
Time with me in view	about 6 h 30	about 9 h
Names taken back	11	177
Per hour	about 2	about 20
Given back within a minute	10 out of 11	169 out of 177

The robot now takes my name back about once every three minutes, and gives it back a few seconds later: 9 seconds in the middle of the pack. Each time, it was almost certainly me.

My best guess, not measured yet: by letting it check my face at every angle, I also gave it many more chances to doubt. Before, a face in profile was never checked, so it was never refused. Now it is checked every three seconds, and three weak scores in a row are enough to take the name back. Those weak scores are not about distance: two thirds of the faces were large in the image. What angle, what light? The log does not say yet, because it does not record the angle when it takes a name back.

Next

Make the log record the angle and the light each time it doubts. Then decide whether it should take a name back at all when the person never left the image. Luckily, it does not say hello every time it finds me again: 36 greetings on Saturday, not 177. But a colleague who forgets who I am twenty times an hour is not quite what I had in mind.