Developers can now create Pixel 2-like portrait effects using DeepLab-v3+

The Pixel 2's camera package is excellent for a lot of different reasons, and one of my favorite features is easily its portrait mode. The Pixel 2 continues to create the best-looking portrait-style shots even with phones like the Galaxy S9 and iPhone X out in the wild, and now Google's doing the unexpected by taking the system that makes it possible and opening it up to other developers.

DeepLab-v3+ is the technology in question, and this essentially gives each pixel its own label to identify objects in the foreground and background. DeepLab-v3+ is being added to Google's TensorFlow development platform, and as such, developers will be able to integrate this same framework into their apps.

Per Google's announcement post on its Research Blog:

Since the first incarnation of our DeepLab model three years ago, improved CNN feature extractors, better object scale modeling, careful assimilation of contextual information, improved training procedures, and increasingly powerful hardware and software have led to improvements with DeepLab-v2 and DeepLab-v3. With DeepLab-v3+, we extend DeepLab-v3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries.

How DeepLab-v3+ picks out subjects from the foreground and background.

How DeepLab-v3+ picks out subjects from the foreground and background.

DeepLab-v3+ isn't the sole item that powers portrait mode on the Pixel 2, but Google notes portrait mode is "an example of what this type of technology can enable." So, while Google isn't necessarily open-sourcing the Pixel 2's portrait mode, it is allowing developers to use the same system that made it possible.

Joe Maring was a Senior Editor for Android Central between 2017 and 2021. You can reach him on Twitter at @JoeMaring1.

  • I think pixels portrait mode is over rated. My kids constantly have a blurry ear or hair or headband or hat. And then I have a part of the picture that is in focus when it should clearly be out of focus. Too unpredictable.
  • I agree. Only had it on the Pixel 1 and was very hit and miss.
    But it can only get better and the twin lens portrait mode is no better.
    But until someone sorts the software what choice do we have?
  • Better than every other phone iPhone X, Latest Galaxy phones etc I have used.
    Pixel 2 nails it better than any other phone.
  • Well, using the phrase "nails it," implies that the goal was absolutely achieved. Judging from experience and some comments on the forums, the Pixel 2 camera comes close to nailing it, but not quite. No one has nailed portrait mode yet on smartphone cameras.
  • Cool news Joe. Can you or someone here differentiate between what is the Google Camera APK that works on the LG G6 and this? We have standalone camera apps in the Play store, it it seems that there would need to be extensive work to make this open source camera tech work across all devices. And then there's the task of optimization for each camera.
  • Man if Google is open to making this open for other manufacturers they must really have a beast on their hands with the Pixel 3. The Google camera apk is already able to drastically improve the pictures of non Google phones like my LG V20 and add portrait mode functionality.
  • For the life of me, I just don’t understand why there’s no open comment section on the article titled “the best android phones you can buy”, Lol 😂 C’mon, it’ll be fun. Be brave and open that baby up. 😂
  • That's cool. How about OEM use auto HDR+ on their cameras too?
  • It seems to me that this is absolutely a camera gimmick and I couldn't care less if my phone had it or not. Same with the anamojis and AR stickers. It looks good on Google though, but its basically giving away very unimportant tech.
  • I prefer the natural bokeh from the full manual mode as it's way more predictable and faster in the end. I guess it's still great to have a software solution as a back up.