Qualcomm Is Cheating On Their Snapdragon X Elite/Pro Benchmarks (semiaccurate.com) 44
An anonymous reader shares a report: Qualcomm is cheating on the Snapdragon X Plus/Elite benchmarks given to OEMs and the press. SemiAccurate doesn't use these words lightly but there is no denying what multiple sources are telling us. [...] Then there were the actual 'briefings' for the X Pro SoC. To call them pathetic is giving them more than their due. The deck was 11 slides, three of which were empty/fluff, five 'benchmark' slides with woefully inadequate disclosure, and two infographic summary slides. The last was the slide below with the 'deep technical' stats [screenshots in the linked article], much of which we told you about last week. And more.
The rest of the 'disclosure' for Snapdragon X Pro was a list of features that all fall under the guise of exactly what you would expect. The rest was filled with deep 'details' like the GPU capabilities of 3.8TFLOPS. That's it. No specs, no capabilities, no nothing. It was truly pathetic. But wait there is more, or less really, with statements like it having AV1 encode and decode. Trivialities like frame rates and resolutions were seemingly not needed for such technical briefs. See what we mean by pathetic? Those 10 cores are arranged how again? That 42MB of cache is what level? Shall I go on about the bare minimum basics or do you get the point now? SemiAccurate was planning to ask Qualcomm about their cheating on benchmarks at the promised briefing but, well, they lied to us and cut us out of the pathetic bits they did brief on. We honestly would have liked to know why they were cheating but we kind of think they will do their usual response to bad news and pretend it never happened like last time. If they actually do explain things we will of course update this article as we always do.
The rest of the 'disclosure' for Snapdragon X Pro was a list of features that all fall under the guise of exactly what you would expect. The rest was filled with deep 'details' like the GPU capabilities of 3.8TFLOPS. That's it. No specs, no capabilities, no nothing. It was truly pathetic. But wait there is more, or less really, with statements like it having AV1 encode and decode. Trivialities like frame rates and resolutions were seemingly not needed for such technical briefs. See what we mean by pathetic? Those 10 cores are arranged how again? That 42MB of cache is what level? Shall I go on about the bare minimum basics or do you get the point now? SemiAccurate was planning to ask Qualcomm about their cheating on benchmarks at the promised briefing but, well, they lied to us and cut us out of the pathetic bits they did brief on. We honestly would have liked to know why they were cheating but we kind of think they will do their usual response to bad news and pretend it never happened like last time. If they actually do explain things we will of course update this article as we always do.
Prove it (Score:4, Insightful)
That's some seriously stupid clickbait chasing just because he didn't like the slides.
He might as well say this proves P=NP.
Re:Prove it (Score:4, Interesting)
The OEMs built prototypes with these chips and got terrible performance. I would believe that the sources are good. The problem is that the lead in to the article (and the summary) focuses on how the benchmarks were cooked, but gives no other details than the real world performance doesn't match. It was a very long article that says "these run slower than they said they would but we don't know what happened."
Re: (Score:2)
Aren't a lot of "benchmarks" sorta bullshit today, like they can really diverge from actual use cases?
I feel like every manufacturer can goose the numbers on synthetics to get the results they want for press releases.
Re: (Score:2)
Good benchmarks are based around a mix of standard use cases packaged up in a way that is perfectly repeatable.
You can definitely tweak performance to favor higher benchmark scores but that can only go so far without actually improving real world use.
Re: (Score:2)
Well no shit. That's a your bad. Maybe you should leave that out, because now it looks like perhaps you are trying to taint perception.
Re: (Score:2)
Modern CPUs adjust their performance based on the available cooling. The more heat that the system can dissipate, the faster the CPU will go, up to some very high limit that no consumer device ever reaches.
So these chips will run better in laptops with active cooling, and in tablets and phones with passive cooling. Maybe the OEMs need to improve their products, or maybe it's just an Intel-style room heater with some bonus computational ability.
Re: (Score:2)
Re: (Score:2)
It screams like the kind of tortured logic someone who has been personally offended would employ.
I think this person is pissed off at QC, and they decided to lash out in an article, stretching what they really had to look as bad as possible.
Re: (Score:2)
It's hard to tell if the accusation is accurate or not. The summary is so full of hyperbolae that I can't bring myself to read the full article.
The summary sounds like what happens when techies leave a sales-pitch meeting behind closed doors. "Did you see those slides? What the hell? They didn't even fill them with techspeak and nonsense numbers. Are they even trying?"
Re: (Score:3)
Qualcomm's shady tactics with Snapdragon X Elite/Pro benchmarks are really disappointing. Transparency should be key in tech. This seriously calls into question their integrity and trustworthiness.
Which puts them on-par with every other tech company. As long as they all keep lowering the bar together, we're stuck dealing with them. This is how we achieve maximum speed on our race to the bottom.
Prove the lack of useful info? (Score:2)
Why isn't Qualcomm proving their product is useful?
Yeah, 'cheating' is inflammatory, but they're not 100% wrong. How would you characterize partial answers designed to hide any useful detail? As if marketing wrote them instead of someone technically competent?
Qualcomm has long been selling dhrystone machines (useless old benchmark today).
Re: (Score:2)
The summary doesn’t mention that they have sources at multiple OEMs who are telling them they’re getting less than 50% of the claimed performance.
Re: (Score:2)
Pathetically they didn't specify if that was for ARM native software and included language which suggests it was not.
Re: (Score:2)
Sounded a lot like the reporters specified that WART was being used for the OEM’s tests.
Cheating on benchmarks? (Score:2)
Re: (Score:2)
At the risk of dating myself, I remember Intel being accused of fudging benchmarks to prove the Pentium 66 was faster than a 486-100 when it really wasn't.
Then there were Apple's creative AltiVec benchmarks.
And back to Intel's P-4 benchmarks that demonstrated how many NOps per second Netburst architecture could really do.
There is nothing new here.
Apple (Score:2, Troll)
Is this the new M4-killer that I keep hearing about?
Or is that a different project?
Re: Apple (Score:2, Funny)
M2 killer. They forgot to lead the target.
Re: (Score:1)
They never specified which Apple chip it would outperform. Turns out it was the G4.
Re: (Score:2)
Is this the new M4-killer that I keep hearing about?
Or is that a different project?
Why is this modded troll?
I just watched a video about Qualcom entering the PC market with a new series of high-performing chips that supposedly rival the latest Apple silicon. I am asking if this is that chip series, or is that something else? It sounds like these... 10 cores etc.
Re: (Score:2)
Is this the new M4-killer that I keep hearing about?
Or is that a different project?
Why is this modded troll?
I just watched a video about Qualcom entering the PC market with a new series of high-performing chips that supposedly rival the latest Apple silicon. I am asking if this is that chip series, or is that something else? It sounds like these... 10 cores etc.
Yes, it is the same chip.
Snapdragon X Elite.
https://www.youtube.com/watch?... [youtube.com]
Cheating how? (Score:4, Insightful)
Re: (Score:2)
"Here is why you should date me... I can see 6 feet high. And my mother likes me."
When you give partial answers designed to hide reality... isn't that pretty close to cheating? Aka marketing instead of being honest?
Re: (Score:2)
isn't that pretty close to cheating?
That depends on a lot of things.
If QC existed in an ecosystem where honest representations of performance with no omitted information were the norm, then yes, they would be cheating.
However, since they do not exist in such an ecosystem, calling their shitty marketing benchmarks "cheating" is equivalent to libel.
Hypocritical. (Score:3)
Yellow journalism [wikipedia.org] at its finest.
Re: (Score:2)
Qualcomm, not Broadcom. And how do you prove lack of details? Other than pointing them out?
If you don't understand technology enough to compare hardware just tell us, instead of claiming someone else didn't do what you can't see they did.
Re: (Score:2)
How do you prove a lack of details?
By reading that there are no fucking details.
Because there aren't.
The claim is based on several fallacious arguments, with zero "details" or evidence that isn't tainted by incompetent handling.
Stay tuned, truth to come soon (Score:2)
The there is this article [arstechnica.com] which suggests the moment for Windows-on-ARM is finally here, probably.
I guess we will see. Should be interesting.
Re:Stay tuned, truth to come soon (Score:4, Insightful)
What does "settings" even imply in this context?
Why the weird part about them accidentally testing with x86 binaries? (assumed, since the "fix" was to use Arm native)
How on FSM's green earth do these people consider themselves qualified to evaluate what is "possible" for some set of "settings" on a new piece of silicon?
This article was a pile of shit. Even if it ends up being completely accurate- the article is still a flaming pile of shit.
Industry standard, unfortunately (Score:2)
Assumption: Intel still to blame for performance (Score:2)
Intel has failed to compete with ARM for laptops, tablets, and phones. They've failed with in-house efforts, and they refuse to let a third party develop something that competes well with ARM. Thus, I assume that Qualcomm is hamstrung, and this is one of the signs of that.
Quick disclaimer: I think I'm up to date on this, but if I'm not, or if I am somehow misinformed, please fill me in! I'd love to learn about the current "state of play" for Windows on ARM.
Intel's in-house efforts have failed to produce la
Re: (Score:2)
Intel has failed to compete with ARM for laptops
The dizzyingly overwhelming majority of all laptops sold are using Intel CPUs.
Laptops and phones- you're correct.
Intel's in-house efforts have failed to produce laptops, tablets and phones that compare favorably with ARM in terms of energy usage and performance.
Performance per unit of energy used.
In terms of performance, Arm (with one notable exception) isn't competitive at all. Not even close.
See: Atom.
Atom isn't a laptop part- it's for much lower power devices.
That doesn't mean it hasn't been put in laptops- it has, but that was a misplacement.
Your overall points aren't terribly wrong, you just shouldn't have included laptops in your "mobile" category, sin
Re: (Score:2)
Your overall points aren't terribly wrong, you just shouldn't have included laptops in your "mobile" category
Tell that to Qualcomm. Of course, this is why they are having so many problems with their new chip.
Re: (Score:2)
Stop being an idiot.
That statement was obviously in the fucking context of the above discussion, which you apparently didn't read.
What's your angle, dude? You're shilling for something, just come out and say it.
not sure about cheating, but vague is true (Score:2)
They just had a qualcomm rep get interviewed on PC World's youtube channel this week. They spent 20 minutes not discussing details about what it could do. It was obvious that the interviewer was asked not to ask about many things. He was struggling to ask questions without breaking the unknown rules agreed to.
The only answer was that 'casual games will work and maybe a few AAA titles'. He mentioned a few workloads like resolve and audio creation. I have no idea if this thing will actually work for soft
Re: (Score:2)
Since the part will only be going into devices running Windows on Arm, they're kind of hamstrung in defending the silicon without also shitting on their ecosystem and partners.
Now, that doesn't mean that the silicon is any good at all.
It just means that even if it is, they're in a sh
Re: (Score:2)
t the questions you want to ask are applying to Windows on Arm, which nothing short of a fire-breathing 12Ghz i9-45000k is going to make performant.
That is definitely a problem with laptop-class ARM CPUs that are available rather than Windows on ARM (compared to Windows on x86/x64). There are plenty of reports that Windows on ARM runs really well under virtualization on Mx series Macs. And Qualcomm is making the same mistake of just trying to make a phone SoC into a laptop SoC without any major changes.
Re: (Score:2)
Windows on Arm runs like shit on M1 as well.
I can literally personally attest to this.
In particular, its x86 emulation is abysmally bad, which in realistic usage, means almost everything you do outside of interacting with the basic OS widgets is abysmally bad.
Re: (Score:2)
Yeah, I don't know why you would use x86 emulation as part of your comparison. I get that there aren't a lot of apps compiled for ARM, but it's hardly the best way to compare.
Re: (Score:2)
It's a terrible way to compare.
And that's the situation they're locked in.
Windows on Arm's x86 emulation is no Rosetta2, unfortunately.
When you add the ~25% cost of virtualizing Windows on Arm on your M* (which is a reasonable cost), you're soon looking at 20-year-old computer performance.
Apple relied on the fact that they had basically leapfrogged everyone else with the M1's performance to smooth the cost of non-native applications over to consumers.
I.e., even running under Rosetta
Re: (Score:2)
In this case, their chip doesn't even run native ARM code competitively with other high-end ARM chips. So running a benchmark compiled for ARM on theirs vs the same benchmark compiled for x86 on an x86 CPU they still fall way short.
Re: (Score:2)
The article is strewn with sheer incompetence.
Apparently they didn't even fucking know they were running an x86 benchmark binary to begin with.
It's unknown what else they have done stupidly.
Re: (Score:2)
He mentioned a few workloads like resolve and audio creation
Two-track audio editing takes now CPU power at all. With on-board encode/decode of common video codecs, a basic video editing task is no different than audio. That really isn't mentioning much - it's not a claim that it can run multi-layered compositing or anything.