This is how the Pixel 4's super-fast face unlock actually works
The Pixel 4 does not have a fingerprint sensor and relies on using your facial features for biometrics — that means unlocking the phone and accessing apps that depend on you proving your identity. A lot of folks aren't happy with the change, but the powers that be at Google want this for its future phones, so that's what we get.
Besides the back and forth about which method is better, the biggest question about face unlock is how it works. While I can't come up with any real answers to why Google dropped the fingerprint sensor, I can tackle this one: face unlock works by using infrared light to make a 3D map of your facial features. And believe it or not, it's not as complicated as you might think.
There is a lot of stuff in the Pixel 4's top bezel. Too much to hide in a slim bezel or a notch, which is why there isn't one. And it's all important stuff if you want to securely unlock a phone using a face.
The render above from Google lists the hardware involved, and outside of the front-facing camera and the audio port, all of it is needed for face unlock. Here's how it plays out.
The Soli radar chip notices that a hand is approaching the phone, which "primes" the display and wakes it up. When it's picked up, the phone detects that, too, using its accelerometer. This is a signal to turn on the rest of the hardware and get ready to scan and make a face unlock attempt. This is a big reason why face Unlock is so fast: it's ready and already working before it gets close to your face.
More: Here's how the Pixel 4's Soli radar works and why Motion Sense has so much potential
Once the phone is in position, the Face unlock flood illuminator turns on. This is an infrared LED that's pretty bright (that's a spectrum of light greater than the red end of the visible spectrum and we can't see it with our eyes). Its purpose is to flood your face in light so that all of it is illuminated. Since we can't see it, our features won't react to the bright light.
While the flood illuminator is turned on, the Face unlock dot projector is engaged. This is also an infrared LED but it has two differences compared to the flood illuminator: it uses a contrasting color (infrared light also has numerous colors even though we can't see them) and instead of a single wide beam it's broken up into thousands of small points. This creates a matrix of dots on your face that are used to define the shape.
With your face illuminated and covered with dots using infrared light, the Face unlock IR cameras grab an image. These cameras have been specially tuned to get a good set of data about how and where the IR dots contrast with the flood illumination. Technically, a "regular" camera can be used to do this but it would be very slow and/or very insecure.
Once a good "image" of your face and the IR lighting is acquired, software magic comes into play. The IR dots projected on your face are not uniform. They are when they exit the projector, but once they meet your face they are skewed and misshapen based on the surface and movement of your facial features. Using this information, the brains inside the Pixel 4 can collect data about the shape of your face and create a token during face unlock's setup. Think of this as a number between one and infinity and you'll have an idea of how it works.
More: How to optimize Face unlock on the Pixel 4
This token is stored inside the phone on the Titan Security Module. It never leaves your phone and nothing can access the data used to create it or the token itself. Every time you use face unlock, a token is created and sent to the Titan Security Module, where it is compared with the original. If the two are a close enough match, face unlock passes the check and your phone or app can be unlocked.
How closely the two must match is based on a couple of things. The closer the match must be the slower it will be. The faster the operation is, the less secure it is. What Google had to do was fine-tune the software algorithm so it was fast enough and secure enough. That's where the real work is.
The process of gathering the data by means of IR light and sensors is actually pretty common. A lot of the things you use and eat every day have been sorted or graded using this type of technology, but when you add in a security factor, things get a lot more complicated. Because most of that process is software-based, we can expect face unlock to get even better over time.
Get the Android Central Newsletter
Instant access to breaking news, the hottest reviews, great deals and helpful tips.
Jerry is an amateur woodworker and struggling shade tree mechanic. There's nothing he can't take apart, but many things he can't reassemble. You'll find him writing and speaking his loud opinion on Android Central and occasionally on Twitter.
"While I can't come up with any real answers to why Google dropped the fingerprint sensor" Its pretty obvious they followed Apple's direction just like they did with the navigation gestures. If and when Apple brings back it's touch ID which is rumored they are then magically Google will bring back the fingerprint sensor option. Face unlock works exactly like Face ID as it has the exact same kind of sensors and cameras. The only thing they added is the soli /radar chip which makes it faster.
Alternatively: Apple purchased the only company that could make the parts small enough to fit into a phone bezel, ensuring it — and only it — could have access to them. Two years later, they are now off-the-shelf parts.
Yeah they bought Finisar I remember and analysts or least one said Apple has about a 2 year lead with the components that make up face id. He was right. Plus there was a limited supply from them and also LG. Nothing wrong with them buying up the company cause that gives them the advantage over all the other companies that would try to source from there too.
BB10 is where Apple stole the gesture based OS
And BB10 copied from Palm. The point is no one bothered to implement those navigation gestures until Apple did. Just like how face unlock is taking off, system wide dark theme all bec Apple has done it or will be.
I've done some very un-scientific tests with the face unlock (out of plain curiosity), and I've found even with one eye and half my mouth covered, the phone unlocks. For whatever reason, my whole nose must be visible or it does not unlock, even if the rest of my face is clear. It seems that the phone is fairly liberal with matching the stored information with what it "sees" when the phone is attempting to be unlocked.
Most likely it gets the bulk of its data from your nose, cheekbones, chin and if visible, your ears. Those are the parts of a face that have the most shape.