Bench Talk

As an editor I’ve been on the receiving end of a lot of specmanship, which is why I always look to datasheets, though meaningful comparisons of seemingly similar MCUs between different vendors are hard to construct, especially since overall energy consumption is so application specific. As Markus Levy of the Embedded Microprocessor Benchmark Consortium (EEMBC) remarked to me last week, “Datasheets don’t lie, they just present [information] the way they want to present it.”

With so much to be gained by presenting your MCU in the best possible light—as an ex-PR Guy I get it—I’m surprised that so many MCU vendors are involved in developing a new benchmark: namely Analog Devices, ARM, Atmel, Cypress, Freescale, Microchip, Renesas, Silicon Labs, Spansion, STMicroelectronics, and Texas Instruments.

This week EEMBC announced EEMBC ULPBench, a standardized benchmark for ultra-low-power MCUs. Levy claims that “CoreMark is too big for the ultra-low-power space, it draws too much power.” So a more compact and meaningful benchmark is called for.

The original release will be ULPBench-CP (core profile), which measures CPU core efficiency as well as the microcontroller’s real-time clock and calendar function (RTCC), and power modes. Subsequent releases will utilize different peripherals in standardized applications.

I applaud EEMBC’s goal of developing “fair and unbiased benchmarks for the embedded industry.” But the devil’s in the details. I can envision a basic benchmark that would measure data throughput of all MCUs in a certain class. But what are the assumptions and constraints? Can I use the data transfer controller (DTC) in a Renesas RL78 to transfer the output of the ADC to RAM? That would save a lot of cycle time but it might not be allowed by a standardized benchmark, in which case the RL78 might not perform as well on paper as it would in practice. The same question applies to processors from Microchip, Atmel, and everyone else on the list with vendor-specific hardware accelerators that would be hard to specify in a general benchmark.

What do you think? Where do you see this possibly getting tricky? And which specific applications would you like to see benchmarked? ULPBench is still evolving, so your feedback would be both helpful and, I’m sure, appreciated.