Without the computer: moving the program onto the robot

Until 20 October, the robot’s eyes were the robot’s, but its brain was my Mac. That morning I set a new rule: I want to be able to use it without the Mac. Pick it up, put it on another desk, and it still knows me.

Inside the machine

The Wireless version has a small computer inside: a Raspberry Pi Compute Module 4, four cores, about 4 GB of memory. It already runs the maker’s software, which drives the motors and serves the camera. My program had to fit next to it.

Before moving anything, I measured. The same two models as on the Mac, image by image:

Step	Mac	Robot
Find faces, image 1,280 px wide	9 ms	404 ms
Find faces, 640 px	4 ms	104 ms
Find faces, 320 px	2 ms	22 ms
Make one fingerprint	6 ms	121 ms

The robot is ten to forty-five times slower. So the program on board looks for faces in a reduced image, and only computes a fingerprint on the small zone where a face was found.

The maker’s software has a mechanism for apps: an app is installed on the robot and can be launched at start-up. Mine became one.

First launch on board

On the first launch, with the Mac closed, it recognised me in 6 seconds and gave 3.3 verdicts per second. With nobody in front of it, it used one and a half of its four cores, 183 MB of memory, and the chip sat at 51.6 °C.

Then the problems, one per test:

1. My face was cut by the top edge of the image for forty seconds: at my desk, I sit higher than the robot’s gaze. Fixed by letting the head follow me from the start.
2. Far away it did not know me. Close up I scored 0.70; further away, with a face only 40 to 60 pixels wide, 0.30 to 0.45, under the threshold. Re-detecting the face at full resolution, inside the box found in the reduced image, gained 0.10 on the score.
3. After each sound, the head did not come back. A recorded movement moves the head, and the head only came back 60% of the way. With a longer return and a short grace period, the sixth test ended with the head back within 4 degrees and a stable score of 0.68 to 0.72.

The antenna that starts everything

Last step: the program starts by itself. After a cold start the robot sleeps. Push an antenna, it wakes up, and my program launches with it. Measured after a cold restart: the maker’s software was ready in 27 seconds, and the robot recognised me 21 seconds after I pushed the antenna.

One limit I cannot fix: the battery has no gauge. A LED turns from green to orange to red, and the charge cannot be read by software.

Next

It worked without the Mac. It also worked too much: a few hours later, it was reacting to every noise in the room.