In light of this morning’s news about Samsung modifying some code to boost their benchmark results for the Galaxy Note 3, giving a 20% increase in scores, I got thinking, are benchmarks THAT important anyway?. This is not the first time they have been “caught” doing so as well; When their flagship Galaxy S4 came out, they also boosted their CPU and GPU clock speeds in hopes for those bigger numbers. Shame on Samsung right? Wrong. Shame on anyone who actually thinks that a phone’s benchmark score on AnTuTu or Quadrant actually means anything when you’re choosing a phone.
Article continues after the jump
This phenomenon is not just confined to the Android ecosystem. When the iPhone 5S first came out, the Internet was rife with praise on how it “slaughters quad-core rivals in performance tests despite dual-core CPU and just 1GB of RAM” According to BGR and Anandtech, 2 very esteemed publications, the Sunspider and Google Octane tests showed that the iPhone beat out all rivals with a fairly wide margin. This is thanks to the 64-bit A7 chip in the new phone, after all it was said to be 31% faster than the A6 and up to 40x faster (according to one of their slides). Well of course it is.
Credit: Engadget Live Blog Coverage
No phone manufacturer in the world would release a new product which is slower than its predecessor. Let’s look for a second at that graph. 40x faster than what, the original iPhone? That’s a terrible comparison at best. BGR and Anandtech are doing awesome work with what they are writing and the tests they do, but I think benchmarks and numbers are doing more harm than good in the world of the smartphone.
Photo Source: VR Zone
Now I’m not calling into question the practice of benchmarking in general because let’s face it, if you’re a PC guy who takes pride in building a PC from scratch whether it be for overclocking or just hardcore gaming, benchmarks are everything. With the tweaking of your clock speed, upping of your voltage or even swapping parts out gives a difference in your overall hardware benchmark and could mean the difference between an unstable 120fps, smooth 80fps or jittery 60.
Smartphones are not desktop PCs. In a desktop PC environment, the experience doesn’t change, the OS is pretty much the same across the versions and the usability is confined to what parts you choose and what input peripherals you choose. This is where benchmarking can determine which PC is better. Higher numbers = better parts for the most part. For a smartphone, where there is way more to the device than just what powers it is a whole different ballgame.
It’s an iPhone 4 but you get the point
Let’s go back to the iPhone 5S example: While its true and undisputed that the phone is obliterating its rivals in Sunspider and Octane, but that speaks NOTHING for how good the phone is. iOS and Android are 2 intrinsically different operating systems and user experiences. How can one compare which is better based on how fast they crunch numbers? Let’s bring the Nokia N9 into the picture. Hardware from 2009 will never beat the newest phones now, but let’s look at the user experience. Some will tell you that the N9 still had one of the first gesture based operating systems which predated lots of modern OS’ features. MeeGo was a truly intuitive operating system, unluckily cut short. Now just because it couldn’t rock a good Sunspider score, does that make it worse than an iPhone? Heck no. Pang sure doesn’t think so.
So the next time you read a review about a phone, try to keep the benchmarks to a “FYI” sort of thing. Whether or not a phone is good is down to which operating system you prefer, if you like the build of the device and nowadays, how well the camera performs. In your time with your current phone, I’m pretty sure no one has ever thought “I wish that my Galaxy S4 had 1000 more points on AnTuTu, that would make it a whole lot better”. Granted it is a good indication of how fast certain applications will open or run, but that in itself is not the be all and end all. Read about the build, first impressions and the gripes my fellow writers and I have with the devices and soon, you’ll have your answer to your questions without even having to worry about a single benchmark.
I’d like to know what you think; If you’re reading the benchmarks first for a smartphone, what is your thought process? Is it purely to see how fast it will run apps? Have you bought a phone purely based on benchmarks? Leave a comment below!
