I would expect the Thumb and Thumb2 results to be similar for a given compiler.
I wonder why ARM wanted their compiler removed from the results? Seems a bit odd.
01 June 2009
ARM Announces Support For EEMBC CoreMark Benchmark
New open source benchmark provides an alternative to Dhrystone
WHAT: ARM today announced their support for the new EEMBC (Embedded Microprocessor Benchmark Consortium) CoreMark processor benchmark.
CoreMark is a synthetic benchmark, but carries out meaningful calculations, and is claimed to be a significant improvement on current Dhrystone benchmarks which is overly dependent on the performance of the C library...
it was outperformed by the other compilers in most tests (with the exception of the full "printf" tests).
There are several reasons why tool vendors do not publish comparing benchmarks. Regardless how thorough you may be when you run benchmarks as a tool vendor, you can by definition never be regarded as being objective. And publishing results where you compare your own tools with the competitors is therefore rude!
I have been in this business for more than 20 years now and I have seen some really horrifying examples where benchmarks have been used in order to deliberately misguide customers. The worst example was a vendor (no names!) which compared tools by using the default settings of the tools. Their own default was maximum optimization turned on and the usage of the smallest possible memory model while the competitors tools had optimizations turned off and used the largest possible memory model by default. Such benchmarks are worse than useless!
So, are we tool vendors using benchmarks or not?
Yes we are using them every day, but in house as a vital part of our development. Benchmarks indicate changes in code size and speed during our development. If the current revision differs in the wrong direction compared to previous revisions, then something may be broken in the compiler. Benchmarks serve in that way as a complement to all the other test suites which are testing conformity and functionality.
The second usage is of course comparing our tools to the competitors so we preferably are the #1 when we release new versions of our tools :-)
Retrieving data ...