Ask Jerry: Is my smartphone or smart home device listening to everything?

Welcome to Ask Jerry, where we talk about any and all the questions you might have about the smart things in your life. I'm Jerry, and I have spent the better part of my life working with tech. I have a background in engineering and R&D and have been covering Android and Google for the past 15 years.

I'm also really good at researching data about everything — that's a big part of our job here at Android Central — and I love to help people (another big part of our job!). If you have questions about your tech, I'd love to talk about them.

Email me at askjerryac@gmail.com, and I'll try to get things sorted out. You can remain anonymous if you like, and we promise we're not sharing anything we don't cover here.

I look forward to hearing from you!

What does always listening really mean?

Rob asks:

I know smartphones and assistants listen for their wake word. Does that mean they hear everything around them all of the time?

Thanks

Hi Rob. That's a great question that I think a lot of people wonder about: how much does their smartphone or "smart" device actually hear?

The easy answer is everything. Yes, Always Listening means just that — it is always listening. It has to so it can recognize when its trigger word or phrase was spoken, then it can try to do something useful. This means that all smart home and assistant-powered devices from every company, including Amazon, Google, and Apple, must work this way.

The better answer is to explain how it's supposed to work, according to the people who make the products and those who have tested them. Always listening doesn't mean always recognizing or always doing something. I'll use a simple Nest Mini as an example, but as mentioned, this is pretty much the same across all ecosystems and platforms. Any sort of AI or assistant on your phone has to work this way.

As soon as you plug in and set up a Nest Mini, its microphone goes to work. Every sound that it picks up is analyzed using some low-power software and hardware, and unless you've said "Hey Google", that's the end of it.

If it does think it's picked up its command phrase, other processing starts to happen. The software tries to interpret the words it has heard into commands the device can recognize, and then tries to act on those commands.

An example is if I say, "Hey Google, turn off the porch light," My Nest Mini catches the phrase Hey Google so it knows it needs to process what comes next. In this case, it uses speech recognition to determine my request, ensures it recognizes a device named "porch light," and then sends a signal to shut it off.

If it didn't recognize the key phrase of "Hey Google," it would just keep listening for it.

If it recognized its key phrase but was unable to figure out what was being asked, or if it were asked to do something it couldn't, it would respond with a generic error like "I'm sorry I don't recognize porch light" or "I'm sorry I can't do that".

If it "thought" it recognized the key phrase, it would try to figure out what to do next as described above. Usually, it can't do anything because we never really asked it to do anything. In that case, it just goes back to listening. But sometimes, it can be goofy and think it found a command and do something you never asked it to do.

I have a first-generation Google Home here that will start playing YouTube videos out of the blue at least three times a week because it thinks I asked it to do it. Goofy little gadget.

What about privacy?

Tech companies are very specific about what their voice-activated devices will do and how they do it. That's good because having something that hears everything could be a little problematic if not kept on a short leash.

For example, some of the commands it "hears" are recorded and sent back for someone to check. If it was told to turn off the porch light, someone wants to make sure it did turn off the porch light. It does keep track of everything it hears that it thinks is a valid request for a certain amount of time. Some think that this information is sold to advertisers, though the companies making the smart devices say otherwise.

If it heard something that wasn't a valid request, it won't keep track of it, according to the companies who make them.

Yes, this is a very valid privacy concern. Most people don't want Google to spy on everything they say. I know I don't. Not that I'm saying or doing something I shouldn't be (you never know) but it's the fact that it's none of Google's damn business.

You really only have two choices: trust that it's doing exactly what it is supposed to be doing by only processing and acting on things it thinks are valid requests, or don't. I can't tell you what to choose here, but if you don't trust it, you should totally not use it.

Myself? I only ever fully trust something I built with my own two hands while I'm using it myself. I do use Google Assistant, Gemini, and Google Home products, but I'm mindful of where, when, and how. I'd never have one in my bedroom, for example. Especially one with a camera. I do not use any voice-activated smart "stuff" through Apple, Microsoft, or Amazon because I only want to worry about one platform getting more than I want to share.

You probably shouldn't follow my example and decide if you trust it or not, then go from there. Giant tech companies don't care about you as a person, but are almost assuredly doing exactly what they describe with voice-controlled services and products. Not to be kind, but because it's not worth the problems that would arise if they got caught doing something differently.