Every year, Qualcomm gives tech media the opportunity to benchmark the newest Snapdragon Reference Device so they have a baseline to compare them to shipping devices that come months later, and what kinds of improvements they can expect over previous Snapdragon platforms.
The Snapdragon 865 is a big deal for a number of reasons, but in terms of straight-up year-over-year benchmark improvements, it's pretty iterative.
The Benchmarks
While in Hawaii for Qualcomm's Snapdragon Tech Summit, me and my colleagues, Andrew Martonik and Nirave Gondhia spent a couple of hours running a set of very specific benchmarks — AnTuTu, Geekbench 5, GFxBench, Androbench, PCMark, Speedometer, Jetstream, and a couple others — so we could glean a few numbers and figure out that, yes indeed, the Snapdragon 865 is faster than the 855.
But that shouldn't come as a surprise. Here are a few of the results comparing the Snapdragon 865 RD to the 855 from the previous year:
| Benchmark | Qualcomm Snapdragon 855 RD | Snapdragon 865 RD | % difference |
|---|---|---|---|
| AnTuTu 7.1.1 | Row 0 - Cell 1 | Row 0 - Cell 2 | Row 0 - Cell 3 |
| Overall | 359346 | 545354 | 34% |
| Geekbench | Row 2 - Cell 1 | Row 2 - Cell 2 | Row 2 - Cell 3 |
| Single core | 604 | 927 | 35% |
| Multicore | 2267 | 3432 | 34% |
| GFXBench 4.0 | Row 5 - Cell 1 | Row 5 - Cell 2 | Row 5 - Cell 3 |
| ES 3.1 Manhattan 1080 offscreen | 71 | 88.68 | 20% |
| ES 2.0 T-Rex 1080 offscreen | 163.54 | 204.85 | 20% |
| Jetstream 2 | 65.54 | 69.42 | 5.6% |
| Speedometer 2 | 64 | 69 | 7.2% |
Whether the benchmarks are CPU- or GPU-constrained, the Snapdragon 865 performs better than its predecessor across the board. That's a good thing, because it shows that year over year gains in performance are obtained despite not ramping up clock speeds but by optimizations to the CPU and GPU designs themselves. In particular, it's noteworthy that Qualcomm relied on an unmodified ARM Cortex-A77 design for the prime and performance cores versus making changes to the Cortex-A76 cores in last year's platform.
Our colleagues at Anandtech took a much deeper dive into the Snapdragon 865 and here's what they said about the CPU system performance:
Performance is exactly where Qualcomm advertised it would land at, and we're seeing a 25% increase in SPECint2006 and a 29% in SPECfp2006. On the integer side, the A77 still trails Apple's Monsoon cores in the A11, but the new Arm design now has been able to trounce it in the FP suite. We're still a bit far away from the microarchitectures catching up to Apple's latest designs, but if Arm keeps up this 25-30% yearly improvement rate, we should be getting there in a few more iterations.
In other words, Qualcomm still falls behind Apple's 2017 A11 processor in some respects but is now ahead in others. But there's another narrative that we haven't mentioned: sustained GPU performance. Qualcomm said that while average GPU performance should be around 20-25% better in most workloads, those workloads shouldn't tax the graphics chip the same way as they did in previous generations.
Anandtech once again:
Overall, we were able to verify the Snapdragon 865's performance improvements and Qualcomm's 25% claims seem to be largely accurate. The issue is that this doesn't seem to be enough to keep up with the large improvements that Apple has been able to showcase over the last two generations.During the chipset's launch, Qualcomm was eager to mention that their product is able to showcase better long-term sustained performance than a competitor which "throttles within minutes". While we don't have confirmation as to whom exactly they were referring to, the data and narrative here only matches Apple's device behaviour. Whilst we weren't able to test the sustained performance of the QRD865 today, it unfortunately doesn't really matter for Qualcomm as the Snapdragon 865 and Adreno 650's peak performance falls in at a lower level than Apple's A13 sustained performance.
Remember me
But one area the Snapdragon 865 will likely delight people, especially when it comes to app load times and general snappiness around the OS, is its improved memory latency. Anandtech again:
One of the larger changes in the SoC this generation was the integration of a new hybrid LPDDR5 and LPDDR4X memory controller. On the QRD865 device we've tested the chip was naturally equipped with the new LP5 standard. Qualcomm was actually downplaying the importance of LP5 itself: the new standard does bring higher memory speeds providing better bandwidth, however latency should be the same, and power efficiency benefits, while there, shouldn't be overplayed. Nevertheless, Qualcomm did claim they focused more on improving their memory controllers, and this year we're finally seeing the new chip address some of the weaknesses exhibited by the past two generations; memory latency.We had criticised Qualcomm's Snapdragon 845 and 855 for having quite bad memory latency – ever since the company had introduced their system level cache architecture to the designs, this aspect of the memory subsystem had seen some rather mediocre characteristics. There's been a lot of arguments in regards to how much this actually affected performance, with Qualcomm themselves naturally downplaying the differences. Arm generally notes a 1% performance difference for each 5ns of latency to DRAM, if the differences are big, it can sum up to a noticeable difference.While it's a very good improvement in itself, it's still slightly behind the designs of HiSilicon, Apple and Samsung. So, while Qualcomm still is the last of the bunch in regards to its memory subsystem, it's no longer trailing behind by such a large margin.
Poor memory latency is not something that comes up a lot in benchmarking, but Qualcomm made it clear that a lot of work was done to alleviate some of the issues from previous generations, and that's without even supporting LPDDR5 memory.
You can call me AI
One of the main areas of improvement in the chip is how it processes machine learning- and AI-heavy tasks. The Snapdragon 865 claims to have an AI engine twice as powerful as the 855, hitting 15 trillion operations per second. That's an astounding number, for sure, but it doesn't always translate properly in terms of real-world performance, mainly because the use cases for these kinds of apps and operations are kind of limited.
XDA-Developers has a really good overview of how Qualcomm eked out such incredible numbers from its fifth-gen AI engine. I like their conclusion, as it sums up why the AI Engine is important, if not now then into the future:
Services that involve translation, object recognition and labeling, usage predictions or item recommendations, natural language understanding, speech parsing and so on will gain the benefit of operating faster and consuming less power. Having a higher compute budget also enables the creation of new use cases and experiences, and moving processes that used to take place in the cloud onto your device. While AI as a term has been used in dubious, deceiving and even erroneous ways in the past (even by OEMs), many of your services you enjoy today ultimately rely on machine learning algorithms in some form or another.
Why these benchmarks are not the whole story
I don't love benchmarks, which is why this article is in the format it's in. I'll be the first to admit other people do it better and more comprehensively than I do, and I choose to be dubious about how synthetic or even organic benchmarks actually translate to real-world experiences in the smartphone space.
I'll give you an example: the Snapdragon 670 found in the $399 Google Pixel 3a does not translate to demonstrably worse performance than the Snapdragon 855-powered Pixel 4. Mid-range is no longer a dirty word, and Snapdragons in the 600 and 700 series are more than capable of getting the job done for the vast majority of people.
As impressive as they are, these are synthetic benchmarks that barely touch the surface of what a commercially released phone is asked to do, and is capable of doing
So does that mean the iterative improvements we see in the Snapdragon 865 aren't noteworthy? Of course not — as workloads get more complicated, they move off the CPU and onto the GPU and DSP (AI Engine), these high-end chips will enable more specialized and less generalized tasks. Think heavy computational photography abilities or real-time translation, things that only trickle down to the 600 and 700 series platforms a year or two after they debut on the 800s.
The synthetic benchmark narrative also doesn't touch on many of the other areas in which the Snapdragon 865 excels: 5G, which comes on all devices (for better or worse), and most notably, camera abilities. Those evaluations will have to wait until we get the first smartphones running the Snapdragon 865, likely in late Q1 with the Galaxy S11 series.
Given the warm but muted reception awarded to the Snapdragon 855 this year, the days of massive annual performance bumps between generations is probably over. Instead, it's the stuff on the edge that's most exciting, and I can't wait to take that picture in the coming months.
