DxOMark revamps its mobile testing for the phone cameras of 2017

The way we use our phones' cameras has changed a lot since 2012, back when DxOMark first started testing smartphone shooters. Until today, the firm's scores only take into account a limited subset of the capabilities of modern smartphones — but that's about to change.

In recent months, DxO has embarked on an extensive data-gathering mission to find out how people use their smartphone cameras in 2017. And as a result it's come up with a new testing and scoring system to more accurately reflect the capabilities of modern phone cameras, and how people use them.

The new DxOMark mobile measures the effectiveness of depth-of-field and zoom features in dual-lens cameras, as well as tracking moving subjects, and extreme low-light testing. DxO's existing testing parameters for main cameras are also to be overhauled as part of its new scoring system, as it hopes to more accurately reflect the quality of photography from modern smartphones.

The Google Pixel now ties the U11 with a DxO score of 90.

The first DxOMark reviews under the new system have been published today — the firm re-tested any models which occupied the number-one spot at the end of recent years (that includes the HTC U11 and 2016 Google Pixel phones, as well as the iPhone 7 and Galaxy S6 edge.) The firm also released its first test of Apple's iPhone 7 Plus, complete with tests around the secondary zoom lens. Bottom line: The U11 and Pixel both score 90 under the new system. So Google's flagship now ties HTC's for the "best rated" smartphone shooter. (That's if you buy into the idea that a numbered score can sum up something as complex as a phone camera.)

Also tested under the new system: the iPhone 7 (85), iPhone 7 Plus (88) and Galaxy S6 edge (82).

Both HTC and Google have publicized their DxO scores over the past year, and under the new system, both devices get to share in the glory — at least for the moment. It's worth noting, however, that newer cameras like the LG G6 and Samsung Galaxy S8 haven't been tested, since they weren't winners at the end of any recent years. It's unclear if DxO will re-test devices which didn't attain top scores under the old system, but leaving out newer devices from Samsung and LG would seem like a substantial oversight.

The idea of reducing a smartphone camera (or now, cameras) to a single numbered score remains a point of contention for some, but DxO insists its new, updated testing method presents the best overall picture of a mobile camera's performance.

More: DxOMark

Alex Dobie
Executive Editor

Alex was with Android Central for over a decade, producing written and video content for the site, and served as global Executive Editor from 2016 to 2022.

  • Retest the Huawei P10 and Mate 9
  • Interesting and maybe telling... "We now take dual camera setups into account... Also, the two phones tied for first place don't have them."
  • Only one phone officially tested so far had that set-up and its score went up by 2 points.
  • So, if that single example can be taken as a trend, I can't wait to see the popular reaction if the Oneplus 5 gets scored an 89 XD. Although in seriousness, they now factor in front facers too, and the one in the Oneplus 5 is apparently really good...
  • I hope it does get a higher score, I am much more impressed with that camera than a lot of the reviewers were.
  • I've certainly had no issues... Not thatI think Oneplus are special... I'd likely be saying the same with any manufacturers top tier phone. Well, except essential and possibly Moto Haha.
  • Interesting. Now Google can tout it's "best ever" score again just as the new models are about to come out. Seems really fishy they would take the time to score the s6 and not the s8 or Note...I'd be curious to see if those numbers are publicized before the Pixel 2 launch.
  • afaik they hadn't tested the Note 8 yet at all.
  • Also, "the firm re-tested any models which occupied the number-one spot at the end of recent years".
  • It is fishy. Clearly Google paid them off to counter Samsung paying off Android Central
  • LMFAO!
  • Clearly. HAHA!
  • They should just give dual camera phones individual scores for each camera. Main camera 89, secondary camera 85, would be easier. But, you have to deal with cameras that use imaging information from both cameras, so I guess it's a mixed bag. Wonder what they would do with quad cameras.
  • I haven't looked, but they probably do, and then average the score. Camera A scores 89, camera B scores 85. Combined score 87. It's that they already did for stills and video.
  • It doesn't separate them. It'll take photos using both cameras at the same time and analyze those shots. In the case of the iPhone 7 Plus, the 2nd camera adds to the strength of zoom and portrait shots. One major downside of the new scoring is that it's not transparent anymore, we can't calculate the score based on the values they give us.
  • Guys before jumping off the conspiracy theory cliff, read this part here: "the firm re-tested any models which occupied the number-one spot at the end of recent years". So if you were #1 in recent years, you got retested in the first wave.
  • Shhh, you'll spoil it!
  • Well it is easy to understand why, tomorrow will be launch of the Iphone X, and with the previous method, the iphone will not surpass the pixel nor the htc u11, the only way to do it is to add zoom and bokeh assessment (u11 and pixel doesn't have it) and make sure that the new iphone surpass them. The level of details, processing, color reproduction... are the most important part in camera results, but it seems that the market/big players needs something else.
  • Thumbs up to them for revamping the scores, but a massive thumbs down for including bokeh effects in the final scores. Even on phones with an extra telephoto lens it looks rubbish compared to an SLR.
  • I wouldn't consider grading on a fake effect valid anyway.
  • They need to adjust so that it continues to be easy to fabricate the scores according to the payments. Of course, their credibility is still non-existent and shall remain so in the future I think.
  • That's not how any of this works. Where do you get this crap?
  • I don't know how anyone could trust DxOMark scores when they give a 90 to a camera that has a known halo flaw. I don't care how many excellent pictures the Pixel might take. The fact that some will have an annoying halo disqualifies it from getting such a high score.
  • They note the lens flare issue on the Pixel, and knocked the score down by a couple points. But, they also tested zoom using the telephoto lens on the iPhone 7, and only digital zoom on the others. Well duh... of course a telephoto lens is going to give you more detail of distant objects. Seems like selective testing to me. Why not test macro mode, or how fast/easy the camera can be launched? I went on a walkabout downtown with a guy who had an iPhone, and he downright missed several shots that I snagged one-handed.
  • The lens flare is overblown.
  • Maybe they shouldn't test them at all. It is still subjective. Does anyone buy a phone because of the camera? If I was on the fence about what phone to buy this might make a difference but if you're that serious about photography you're not using a smartphone for pictures in the first place.
  • Wow, one of the best things HTC had going for it was best camera, and they took it away. All I care about is image quality. If they're giving points for bells and whistles, then the "test" is pointless.