IBM® has been steadily adding improvements to the IBM Software Developer Kit (SDK) and IBM Java™ runtime from release to release, to enhance Java application performance. Customers using older versions of IBM software, which consequently use an older release of the IBM Java runtime, are missing an opportunity to take advantage of these performance improvements.
Most of these performance improvements have been platform neutral and so benefit Java applications running on all platforms, including Intel Systems, IBM Power® Systems, and IBM z Systems®. However, IBM Runtime developers have been designing new hardware features into newer generations of IBM Power and z Systems to target Java performance opportunities. These new hardware features are exploited only in the newer versions and releases of the SDK.
Typically, Java customers are quick to move to newer hardware levels, but keep running older versions of software and the IBM Java runtime, thus missing out on the performance afforded by the exploitation of the newer hardware in our later releases. In this blog, we will demonstrate how different releases of the SDK perform on recent hardware and software levels and we hope that this discussion will encourage the adoption of newer releases. Any references to Java versions in this article are to IBM SDK, Java Technology Edition.
For all of the performance measurements discussed, we varied only the Java version, keeping the rest of the hardware-software configuration fixed. The release-to-release comparisons were done on the 64-bit versions of the SDK running on Linux on Intel Sandybridge, IBM POWER8 and zEnterprise® EC12 (zEC12) Systems, as well as z/OS® running on a zEC12 System. We have used the latest service refreshes of each Java release. Because IBM SDK developers typically backport low risk performance improvements into service refreshes of the older releases, the performance measurements shown generally represent the best possible performance on each major Java release thus far. We have gone as far back as Java 6.1 (first released in February 2011), so the performance improvements discussed here are from Java performance enhancements over the last five years.
The following legend is used in the comparisons:
IBM WebSphere Liberty 220.127.116.11 server, running the Apache Daytrader3 benchmark, which simulates stock trading, has doubled in performance since Java 6. The Liberty 18.104.22.168 server uses Java 7 by default, but customers are recommended to change their configuration to use Java 8 for optimal performance and the latest hardware exploitation.
IBM tWAS (traditional, or classic, WebSphere Application Server) server running Apache Daytrader3 has improved by over 60% since Java 6. tWAS 8.5.5.X uses Java 6.1 by default, but customers are recommended to use Java 8 for optimal performance with tWAS 22.214.171.124 or later.
The Databricks benchmark on Apache Spark, running on Intel “Haswell” processors, shows a 2.76 times improvement, based on geometric mean, with the latest Java 8 service refresh (SR2) compared to Java 6. Noting that the IBM team have only been measuring and tuning Apache Spark since early 2015, each Java 8 service refresh is expected to further improve Spark Performance.
The IBM Operational Decision Manager (ODM) rules engine has also seen significant (40%-60%) performance improvements from Java 6 to Java 8 on the large rule-set benchmark.
For the small rule-set benchmark, the IBM ODM rules engine showed throughput degradation from Java 6 to Java 7 on some platforms. However, it has steadily improved to surpass baseline performance on all platforms since Java 7 Release 1 (7.1).
So far, we have discussed release-to-release improvements in throughput scenarios. Even though throughput is the most important performance metric, other user experience metrics such as start-up time, ramp-up time, and footprint are also important, especially for application servers (both in native and virtualized environments). The IBM Java runtime has a long track record of excellence in these metrics.
The following charts show that Liberty and tWAS start-up times were significantly improved from Java 6 to Java 6.1 due to ahead-of-time (AOT) compiled code improvements in the SDK. Thereafter, there have been 15-20% start-up improvements in each release. However, these improvements have been obscured in recent releases, mostly because the IBM J9 interpreter was rewritten in C (instead of assembly), and Java security fixes have added extra checking at start-up.
Reduction in footprint size has been sporadic. As with the reduction in startup time, footprint reduction has been mitigated by unavoidable regressions:
- The optimization of footprint gains often conflicts with start-up and throughput gains, so some trade-offs have to be made.
- New functionality typically increases the amount of code and hence the executable file size.
- New Java runtime releases typically incorporate newer releases of the C compiler, which usually increases executable size.
Because the IBM Java runtime environment includes a just-in-time (JIT) compiler, which performs on-the-fly compilation of performance-sensitive methods while the application is executing, the JIT compiler is usually compiling much more actively at start-up. JIT compiling therefore particularly affects start-up and ramp-up of application servers.
The following charts show that CPU time consumption for JIT compilation during Liberty start-up has increased significantly over the Java releases due to a focus on improving throughput. For example, more aggressive method inlining has yielded a big throughput improvement on the WAS Daytrader3 benchmark, but it has led to a four-fold increase in compile time between Java 6 and Java 7.1.
In Java 8, JIT inlining heuristics were tuned to significantly reduce compile time consumption without degrading throughput. The use of Runtime Instrumentation (RI) technology in IBM Power8 and zEC12 to replace software-based sampling, yielded a further significant reduction in CPU time consumption at start-up.
Some of these improvements were also back ported to Java 7.1, leading to lower compile time in service refresh 3 (126.96.36.199).
In summary, we have highlighted the performance improvement in later versions of the IBM SDK. The results described clearly show that newer releases of the SDK include significant performance improvements and are better optimized for the latest hardware. Over the last 5 years, each IBM SDK release has added significant improvements in throughput without sacrificing other user experience metrics such as start-up, footprint, ramp-up, and CPU-time, with their commensurate cost reduction, through efficient use of machine resources.
To take advantage of these improvements we urge IBM Java technology users to adopt the newest SDK releases.
Please post your feedback and questions, or contact us directly:
- Vijay Sundaresen (email@example.com), Technical lead for performance, IBM Runtimes
- Kishor Patil (firstname.lastname@example.org), Manager, IBM Runtimes