System News
Same Server, Three Different SPECpower_ssj2008 Results
Blog Explains Sun Netra X4250 Server Benchmark Performance
July 10, 2009,
Volume 137, Issue 2

demonstrates the difference in power consumption between a small configuration optimized for a power-performance benchmark and a typical configuration optimized for customer workloads
 

The recent SPECpower_ssj2008 benchmark figures that reported outcomes of three identical Sun Netra X4250 Servers running the same software stacks but with different results are interpreted in a blog by Christopher Boire of Sun's Strategic Applications Engineering group.

All the tested configurations had the following components in common:

  • System: Sun Netra X4250
  • Processor: 2 x Intel L5408 QC @ 2.13GHz
  • 2 x 658 watt redundant AC power supplies
  • redundant fans
  • standard I/O expansion mezzanine
  • standard Telco dry contact alarm

Similarly the same software stack was running on each:

  • OS: Windows Server 2003 R2 Enterprise X64 Edition SP2
  • Drivers: platform-specific drivers from Sun Netra X4250 Tools and Drivers DVD Version 2.1N
  • JVM: Java HotSpot 32-Bit Server VM on Windows, version 1.6.0_14

The first result, Boire explains, utilized the Sun Netra X4250 platform with its configuration minimized to the greatest extent possible. In this case, a "Tiny Configuration" was employed, consisting of:

  • 8 GB of Memory (4 x 2048 MB as PC2-5300F 2Rx8)
  • 1 x Sun 146 GB 10K RPM SAS internal drive

Boire writes that this was called the tiny configuration because it seems unlikely that most customers would configure an 8-core server with only one disk and only 1 GB available per core. Nevertheless, from a benchmark point of view, this configuration gave the best result, he explains.

The more typical configuration employed in the other two benchmarks consisted of:

  • 32 GB of Memory (8 x 4096 MB as PC2-5300F)
  • 4 x Sun 146 GB 10K RPM SAS internal drives
  • 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)

Nothing special was done with the additional components, Boire reports, noting that the added memory increased the performance component of the benchmark. The other components were installed and configured but allowed to sit idle, so consumed less power than they would have under load.

One of the intended points in this benchmarking exercise, Boire points out, is to demonstrate the difference in power consumption between a small configuration optimized for a power-performance benchmark and a typical configuration optimized for customer workloads. Hardware (power consumption) is only half of the benchmark - the other half being the performance achieved by the System Under Test (SUT), he writes.

Identical tunings were applied at the software level in all three benchmarking exercises: identical java command-line arguments and JVM-to-processor affinity.

Also applied, in the case of the better results, was the common (but usually non-default) BIOS-level optimization of disabling hardware prefetcher and adjacent cache line prefetch. These optimizations are commonly applied to produce optimized SPECpower_ssj2008 results but it is likely that many production applications would benefit from these settings. So to demonstrate the effect of this tuning, the final result was generated with standard BIOS settings.

In order to avoid the charge of making things look better than they were in fact, Boire continues, the number of JVMs was increased in the typical configurations to take advantage of the additional memory populated over and above the tiny configuration. Additional performance was achieved but it doesn't compensate for the higher power consumption of all that memory, he notes.

The tunings were as follows:

  • Tiny Configuration: non-default BIOS settings
  • Typical Configuration 1: non-default BIOS settings; additional JVMs to utilize added memory
  • Typical Configuration 2: default BIOS settings; additional JVMs to utilize added memory

At the OS level, all tunings were identical, he writes.

Boire concludes that the measurement and reporting methods of the benchmark encourage small memory configurations. Comparing the first and second result with the addition of more memory yielded minimal performance improvement (from 244832 to 251555) but a large increase in power consumption, 68 watts at peak. Small configurations yield the best results on this benchmark, he concedes. On the more typical system, the benchmark overall metric decreased from 600 overall ssj_ops per watt to 478 overall ssj_ops per watt, despite Sun's best effort to utilize the additional configured memory. On typical configurations, reverting to default BIOS settings resulted in a significant decrease in performance (from 25155 to 229828) with no corresponding decrease in power consumption (which was essentially identical for both results).

Configurations typical of customer systems (with adequate memory, internal disks, and option cards) consume more power than configurations which are commonly benchmarked, Boire notes, while providing no corresponding improvement in SPECpower_ssj2008 benchmark performance. The result is a lower overall power-performance metric on typical configurations and a lack of published benchmark results on robust systems with the capacities and redundancies that enterprise customers desire.

More Information

Interpreting Sun\'s SPECpower_ssj2008 Publications - Boire's blog entry

Standard Performance Evaluation Corporation (SPEC)

Sun Netra X4250 Server [...read more...]

Keywords:

fullsource
 

Other articles in the Performance section of Volume 137, Issue 2:

See all archived articles in the Performance section.



News and Solutions for Users of Solaris, Java and Oracle's Sun hardware products
Just the news you need, none of what you don't – 42,000+ Members – 24,000+ Articles Published since 1998