Machine Learning Inference Performance

AIMark 3

AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

In AIMark 3, the benchmark uses each vendor’s proprietary SDK in order to accelerate the NN workloads most optimally. For Qualcomm’s devices, this means that seemingly the benchmark is also able to take advantage of the new Tensor cores. Here, the performance improvements of the new Snapdragon 865 chip is outstanding, posting in 2-3x performance compared to its predecessor.

AIBenchmark 3

AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.

We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.

AIBenchmark 3 - NNAPI CPU

We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.

AIBenchmark 3 - 1 - The Life - CPU/FP AIBenchmark 3 - 2 - Zoo - CPU/FP AIBenchmark 3 - 3 - Pioneers - CPU/INT AIBenchmark 3 - 4 - Let's Play - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/INT AIBenchmark 3 - 8 - Blur iT! - CPU/FP

Starting off with the CPU accelerated benchmarks, we’re seeing some large improvements of the Snapdragon 865. It’s particularly the FP workloads that are seeing some big performance increases, and it seems these improvements are likely linked to the microarchitectural improvements of the A77.

AIBenchmark 3 - NNAPI INT8

AIBenchmark 3 - 1 - The Life - INT8 AIBenchmark 3 - 2 - Zoo - Int8 AIBenchmark 3 - 3 - Pioneers - INT8 AIBenchmark 3 - 5 - Masterpiece - INT8 AIBenchmark 3 - 6 - Cartoons - INT8

INT8 workload acceleration in AI Benchmark happens on the HVX cores of the DSP rather than the Tensor cores, for which the benchmark currently doesn’t have support for. The performance increases here are relatively in line with what we expect in terms of iterative clock frequency increases of the IP block.

AIBenchmark 3 - NNAPI FP16

AIBenchmark 3 - 1 - The Life - FP16 AIBenchmark 3 - 2 - Zoo - FP16 AIBenchmark 3 - 3 - Pioneers - FP16 AIBenchmark 3 - 5 - Masterpiece - FP16 AIBenchmark 3 - 6 - Cartoons - FP16 AIBenchmark 3 - 9 - Berlin Driving - FP16 AIBenchmark 3 - 10 - WESPE-dn - FP16

FP16 acceleration on the Snapdragon 865 through NNAPI is likely facilitated through the GPU, and we’re seeing iterative improvements in the scores. Huawei’s Mate 30 Pro is in the lead in the vast majority of the tests as it’s able to make use of its NPU which support FP16 acceleration, and its performance here is quite significantly ahead of the Qualcomm chipsets.

AIBenchmark 3 - NNAPI FP32

AIBenchmark 3 - 10 - WESPE-dn - FP32

Finally, the FP32 test should be accelerated by the GPU. Oddly enough here the QRD865 doesn’t fare as well as some of the best S855 devices. It’s to be noted that the results here today were based on an early software stack for the S865 – it’s possible and even very likely that things will improve over the coming months, and the results will be different on commercial devices.

Overall, there’s again a conundrum for us in regards to AI benchmarks today, the tests need to be continuously developed in order to properly support the hardware. The test currently doesn’t make use of the Tensor cores of the Snapdragon 865, so it’s not able to showcase one of the biggest areas of improvement for the chipset. In that sense, benchmarks don’t really mean very much, and the true power of the chipset will only be exhibited by first-party applications such as the camera apps, of the upcoming Snapdragon 865 devices.

System Performance GPU Performance & Power
Comments Locked


View All Comments

  • Bulat Ziganshin - Monday, December 16, 2019 - link

    Unfortunately the first page lack latency picture for Core i9
  • Ryan Smith - Monday, December 16, 2019 - link

    Whoops! Reload and try it now.
  • shabby - Monday, December 16, 2019 - link

    Performance looks good but I'm really wary of the external 4g/5g chip and its additional antennas and how they'll affect battery life. Not going to buy any new sd865 phone until reviews pop up.
  • eastcoast_pete - Monday, December 16, 2019 - link

    That's the elephant in the room! Haven't seen any good real-world data on just how much power 5G use will actually add. Faster data are nice (in principle), but if 5G cuts battery life by 30-40% vs. current 4G LTE , pretty much useless. If anyone here has any links to such tests, please post - Thanks!
  • Kangal - Friday, December 20, 2019 - link

    Well, I think generally they're going to stick the 5G SoC next to the QSD 865 SoC.
    That means heat from one will affect the other, and so OEMs will potentially require a larger phone (and larger screen = more drain), but with a smaller area for the battery.

    Not to mention, there is going to be significant battery life hit when using an external radio chipset instead of an integrated one. Remember the huge power savings we saw going from the QSD 600 to the QSD 800 back in 2013. So it will be kind of reversed.

    I think overall, what will happen is that all the improvements in the battery technology and the Cortex A77 are going to be nullified. So phones from 2019 which had the best battery life and performance, are going to see a "side-grade" compared to 2020 flagships. So I believe in essence, the QSD 865 will be much less competitive against the Apple A13 and Exynos 990... compared to the QSD 855 against the Apple A12 and Kirin 980.

    ...I do believe more in-depth benchmarks and reviews will come within 3 months (and validate my hypothesis).
  • MrCommunistGen - Monday, December 16, 2019 - link

    Based on the initial performance estimates from the Cortex A77 announcement article Andrei published back in May, I was actually really excited to see the next generation Snapdragon 865. The lack of performance uplift in the real-world web metrics with the QRD865 is a bit underwhelming, but my main concern is the requirement of an external modem.

    I don't have any experience with the current crop of 5G external-modem devices, but back in the day I used several Qualcomm APQ external-modem devices. Back then battery life being terrible was basically a given since batteries were small and Android was pretty bad at power management, but in hindsight I'm sure at least some of it was due to the external modems...
  • tuxRoller - Wednesday, December 18, 2019 - link

    The disappointing web improvements were also seen last year (

    "The web-based benchmarks such as Speedometer 2.0 and WebXPRT 3 showcase similar relatively muted results. Here I had expected Qualcomm to perform really well given the scheduler performance showcase of the Snapdragon 845. The results of the Snapdragon 855 are quite meagre, especially in a steady-state throughput workload such as Speedometer 2.0. Here the Snapdragon 855 only manages to showcase a ~17% improvement over the last generation, and also lags behind the Kirin 980 by some notable amount."

    However, compare those 855qrd to the numbers seen in this article with actual shipping devices. Pretty big difference.
  • generalako - Monday, December 16, 2019 - link

    >Not going to buy any new sd865 phone until reviews pop up.

    Shouldn't really ever do that anyhow...
  • Kishoreshack - Monday, December 16, 2019 - link

    Don't know why the Performance looks disappointing in web browsing test
  • Kishoreshack - Monday, December 16, 2019 - link

    What is the reason is it performs worse than last years 855 chip
    are we gonna see same kind of implementation in real world devices?

Log in

Don't have an account? Sign up now