Welcome to the age of Soli. Google has shown us how the Soli Sensor will do a couple of neat tricks in the Pixel 4, and while waving your hand like a Jedi to change what song is playing is pretty self-explanatory (and honestly, not that exciting), using it in tandem with dedicated hardware to unlock your phone with your face is something we haven't seen before.
Facial unlocking, however, is something we've seen before. First debuting on Android 4.0 Ice Cream Sandwich with the Galaxy Nexus, using your phone's selfie camera — or, in newer iterations, infrared cameras— to unlock your phone is an incredible experience. It's also fairly fast, easy to use, and until Apple developed Face ID on the iPhone X, extremely insecure. Face unlock that only uses a camera to capture data on a two-dimensional plane is easy to fool with a photo. That made it more of a convenience feature, one even Google said shouldn't be used if you value security.
I've previously written about the tech that drives Face ID; those same words apply to Google's implementation on the Pixel 4. That's not surprising as it's well-established technology that can map and recreate a 3D image of basically anything. The only amount of detail captured is only limited by how long you want to wait for the results; the more data captured, and the more time allowed to process it, the better and more thorough the results can be. But most people don't want to wait that long to unlock their phones, which is why companies need to compromise between speed and security.
A quick refresher on how Face ID works: using a special LED that "washes" your face in infrared and near-infrared light, along with a special component that projects a matrix of contrasting IR "dots" on your face, cameras collect everything in front of it and build a 3D map. This map is then converted into a special token based on the shapes present (usually a face, or part of it) and the token is used to check against the data stored inside a secure element in the phone. If the two signatures match, the phone unlocks.
That means the token can do more than unlock your phone. Apple has integrated Face ID to replace the fingerprint sensor, and Google says the facial mapping from the Pixel 4 will also be able to authenticate you as needed, too. So far, Face ID and Google's facial unlocking as seen on the Pixel 4 seem to be very much the same animal. A quick look at the hardware, though, and we see Google has integrated a lot more into that bezel — and it's all courtesy of the Soli Sensor.
How Soli works
Saying that the Soli Sensor is tightly integrated with facial unlocking and gesture recognition isn't just speculation. Google makes it very clear that the two aren't separate the Soli chip is a major component of the whole system. From the blog post:
This works because of what Soli is — a small and power-efficient RADAR transceiver that has a very fine resolution. That means it can detect a thing that's moving and notice very fine details in real-time versus being able to detect a thing that's very fast with less detail.
I've seen Soli in action and it can easily and reliably detect the slightest tapping of a finger, or the movement of fingers acting as if they're turning a miniature dial or knob. While these were controlled demos under the very best conditions, noticing a moving phone approaching the face should be easy to pull off. You don't need a fine resolution to do that and a simple proximity sensor does something similar.
Another big difference between Apple and Google's implementation of 3D facial recognition is that Google is using two face unlock IR cameras. These cameras are what reads the projected IR matrix and turns the contours and curves into a secure token. Apple only uses one of these cameras in Face ID. There could be several reasons for this: Apple is using better parts, Google is building a stereo image, or maybe a second sensor is needed so that the phone can work when turned 180-degrees on its head. Likely, all three of these things are true. It also solves the issue of unlocking the phone while it's sitting on a table, something Face ID still struggles with.
Google calls these "face unlock IR cameras" but they are also used for any motion recognition the Pixel 4 will support. To cover everything happening in front of the phone, having a camera on each side makes perfect sense. You'll never be able to be perfectly aligned in the front dead center of a camera on your phone when you wave your hand to perform an action. A single infrared camera is able to pick up most of your motions, no matter how far away from center they may be, but adding a second can do a lot to make the system more reliable. And unless it's reliable, Google's Motion Sense is nothing more than a gimmick.
The same applies to facial unlocking, especially any "smart" algorithms Google has working to get better at recognizing facial features. Two cameras can make a stereoscopic view of a facial map and collect a lot more data, or they can work with each other to double-check for accuracy, making facial unlocking faster and less able to be fooled. Having two cameras also means that when the system starts "looking" for your face to see if it should prepare to unlock, it can do so much earlier because it doesn't have to wait for your face to be closer to the center.
Finally, because the tech has matured enough to find its way into a small battery-powered device like a phone, there's a good chance these "face unlock IR cameras" aren't as expensive as the ones Apple sourced for the iPhone X. A less expensive sensor that can't work quite as fast or has a lower resolution is perfectly fine if you're adding a second one to the mix.
Google says that the new Motion Sense feature of the Pixel 4 will be able to "skip songs, snooze alarms, and silence phone calls, just by waving your hand", Admittedly, that's pretty cool but recognizing a waving hand isn't something we haven't seen before.
Besides the LG G8, we can go back to the Moto X to see hand gestures in action. Using a set of sensors placed in the front of the phone, a simple wave of the have was recognized even when the screen was off. Google's ATAP (Advanced Technology and Projects) team was born with Motorola and are responsible for Motion Sense so they have experience here.
Motion Sense won't be using the hardware in the Pixel 4 to its full extent because there is no need to draw a three-dimensional map of your hand each time to want to use it. But it will be using the Soli chip and some smart algorithms. That means what we've seen so far from Google is probably only the beginning.
With actual fine-detail RADAR to sense very intricate movement, the only limiting factor of Motion Sense will be digesting the information it collects. Once algorithms are able to accurately identify other gestures I expect Google to build them into Android on some level and hopefully open an API so third-party developers can get data from the Soli Sensor.
How accurate Motion Sense will be and how often it sees a "false positive" where a fluttering curtain or your cat triggers a gesture remains to be seen. Like any facial recognition, expect bumps in the road.
But will it suck?
Probably not. But it all depends on what you're expecting.
Current face unlock solutions on Android usually rely on the front-facing camera, which only captures only a 2D image. It's fast but also easy to fool, and it's why Google doesn't let device makers use it as a way to pay for things or do anything other than unlock your phone. Other solutions, like Samsung's now-defunct iris scanner or Huawei's spatial-mapping solution on the Mate 20 Pro, are similar to Face ID and Google's upcoming solution because they're considerably more secure.
They're not just matching your face to a known image — they're creating and reading a spatial map.
What that means is the "image" does a lot more than say, "Hey, this is Jerry. Go ahead and unlock his phone," when it sees my face or a photo of my face. Modulated acquisition of spatial distortion maps — a technical term that means "checking out all the bumps, curves, and lines of a thing and saving all the data about what you see" — takes time. Comparing the data collected to a known good sample also takes time. It's also an order of magnitude more secure than looking at and parsing the photo of a face.
Like Face ID in its beginnings, there will be bumps in the road. Google is trying to stave off one of them by collecting facial scans of people on the street because this sort of tech has a tough time with people of color. That's something an AI can learn to work around, and hopefully Google scanning all those faces helps make it happen reasonably quickly. Other issues will be ambient lighting, detection of cosmetic changes like plucked eyebrows of shaving your mustache, and last but not least, glasses. All of these problems are "easily" sorted out, but then the fixes have to be done in a way that doesn't make the system less secure.
My prediction is that Reddit will explode with how bad face unlock on the Pixel 4 is about a week after the phone gets into people's hands, then slowly simmer down as these problems get addressed in software patches. Once algorithms are tuned and sensors are dialed in, other manufacturers can look int incorporating the tech into their own Android models.
At first it may be bad. All tech is. But secure facial mapping has a solid background behind it and is used every day by countless other companies for many other purposes. Tuning it to work on a power-limited handheld device that reads your face won't be an insurmountable problem.
Are fingerprint sensors the new headphone jack? Being replaced by a higher tech solution that may not be as good.
Headphone jacks weren't replaced with anything like that, phones that have headphone jacks still have Bluetooth, headphone jacks simply got removed without being replaced. Although I still don't miss the headphone jack much on my Pixel 3, and I can't think of any reasons why face unlock would be worse.
You surely must of meant secure? "until Apple developed Face ID on the iPhone X, extremely insecure"
Iris scanning was secure and was out on Samsung phones before Face ID.
Yes I know it was.
Already explaining away the bad press you know it's gonna get. Lol
Nope. I won't be using it because my face is my identity, not my password. Same with fingerprints. I can tap out my password in a second or two and not worry about it. I'm just taking Google's little photo of the top bezel and trying to explain what they are (probably) doing because this is a subject I'm very familiar with.
MS had this on the Lumia well before Apple had it on the X. If we expand beyond phones, the Surface family had it before the X. And it was/is as secure as anything Apple has done.
No they never. Lumia had Iris unlock. Big difference
HTC's 3D face mapping takes just as much effort to hack as Apple's Face ID. I thought you guys would have known that, since like, it was in the news...
Did anyone notice rest of the picture provided by Google showing the camera array? I strongly believe they will show off some kind of handwriting air gesture feature for notes. Maybe exclusively to the XL version. Soli can read even the finest of hand movements. Could be a clunky feature at first but pretty cool to show off it's true capabilities. Google is all in with the Pixel 4 and so am I.
They already have?
Eh this is probably gonna be gimicky just like it was on LGs last device. And I could care less about face ID personally.
I was planning on grabing the Pixel XL 4 this year to replace my Pixel XL but since this is new tech, I might hold off until the Pixel 5.
"Other phones require you to lift the device all the way up, pose in a certain way, wait for it to unlock, and then swipe to get to the homescreen."
That's all technically true, but in practice, on an iPhone, that is essentially one fluid motion. I pick up my phone, hold it up in a manner practical for looking at it (which I was planning to do anyway), and swipe up. I don't really have to wait for it to unlock as invariably it is unlocked by that point. I in fact sometimes don't want to go to the home screen. There are times when I just want to see the notifications, which don't require me to swipe up. That 'pose a certain way' is really quite forgiving, BTW.
Not denigrating whatever Google brings out. Just suggesting they make something demonstrably better, rather than try to convince us that what we currently have is awful, making their equivalent seem to be an improvement.
P.S. I own and use a Pixel 3 as well as an iPhone X. I like them both.
Yes, I have the iPhone XS and the HTC U12+, and it's pretty fluid on either one. I'd say the HTC is slightly easier because they put the haptic power button where your thumb or fingers naturally fall, and the click is a little easier than a swipe (to me at least), but they both work well.
Face unlock that only uses a camera to capture data on a two-dimensional plane is easy to fool with a photo. That made it more of a convenience feature, one even Google said shouldn't be used if you value security.
Great article. And, given Google's history with new (for them) technologies at product launch, there will certainly be the predicted issues and scorn from reviewers. Only disagreement I have is about drawing the conclusion that "a less expensive sensor that can't work quite as fast or has a lower resolution" based on the supposition that "there's a good chance these 'face unlock IR cameras' aren't as expensive as the ones Apple sourced for the iPhone X." It is extremely likely that the IR cameras are less expensive, as they are likely available from more suppliers, in use in a wider variety of solutions (thinking outside strictly the mobile phone industry), and cost of production has dropped in the nearly 2 years that have passed since iPhone first integrated them in a phone. But I would argue that while Google could opt for far less expensive modules that aren't as fast or have lower resolution, it's easily as if not more likely that the cameras they use could be less expensive and have similar or even better respective specs.
Apple doesn't just source the tech that powers faceId found in iphones. It owns the company responsible for the first generation Kinect sensor from which the tech is derived. FaceId sensors are just like those found on the kinect, only miniaturised to fit in a much smaller frame.
If Pixel 4 is not coming with wide angle lens, I won't buy it, don't give a rats ass about soli crap
It's 3 cameras. Standard, wide, tele.
Hey, maybe someone can help me with this because I haven't found an answer. I've got like twenty apps, including my password manager than use a fingerprint reader. Does this Face Reading feature work out of the box with all these apps? Or is this something completely different and are apps going to have to be reprogrammed to accept this feature? If it is a matter of reprogramming I'm not going to hold my breath, this is the only Android device that has this tech and Google has NEVER sold a lot of Pixel phones. If this phone is breaking the fingerprint unlock for my apps, it is a nonstarter.
Does this Face Reading feature work out of the box with all these apps? It can. However, the developers of the apps that you're concerned about will have to make their apps work with the "face reading feature". Also, you have to own a device whose implementation of the face unlock feature meets the standard for a secure face unlock. As far as I know, on the Android front, the Huawei Mate 20 Pro, LG G8 and the upcoming Pixel 4 seem to be the devices that meet the standard. I may be wrong but I don't believe Samsung's implementation, using the iris scanning method for face unlock, meets the standards set by Google to allow for it to be used for mobile payments. Hell, Samsung wouldn't even encourage its users to use their iris scanning for Samsung Pay.
Or Soli is just a gimmick and hyped for nothing. Its gestures work in a similar way to the Z-Camera of the G8. The difference?
Google uses a miniaturised radar sensor that uses, well, radar waves and converts them into signals or waveforms that the processor inside the device can understand, and then execute the action that is mapped to a specific gesture. Now face unlock is what I'm interested in and i really want to see how google's implementation will differ to apple's. We'll discover that. In due time of course!!!
A bit early to compare apple and google technology, you have to wait for the pixel 4 and the next iPhone to have a fair comparison. And hand gesture will mostly be a gimmick if it can’t do at majority of the tasks without touching the screen...
It's cool that native facial biometrics was added in Android 10. If Google's implementation of facial unlock on the Pixel 4 is supposedly secure enough for mobile payments, I see no reason why the LG G8's implementation won't be (other than the fact that LG may not release Android 10 for the G8 till probably July 2020).
Get the best of Android Central in in your inbox, every day!
Thank you for signing up to Android Central. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.