Startup time, ramp-up time, and throughput

You’ve probably heard those jokes that start, “If my car behaved like software…” – including that infamous exchange between Bill Gates and General Motors twenty years ago. It’s an urban myth, of course, but fun nevertheless [1]:

Gates: “If GM had kept up with technology like the computer industry has, we would all be driving $25 cars that get 1,000 miles to the gallon.”

GM: “And if General Motors had developed technology like Microsoft, we would all be driving cars that crash twice a day.”

And that retort was followed by a list of other unfortunate “characteristics” of this thoroughly modern vehicle, including a reminder to press the Start button to shut off the engine when you’re done!

Well, maybe the parallel between computers and cars is best avoided.

Or maybe we can give it another try to help us think about the performance of Java: in particular, the Eclipse OpenJ9 Java Virtual Machine (JVM) technology used for running Java programs.

Of course, “performance” means different things to different people and gets measured in different ways. Interestingly, the same complications seem to apply as much to cars as to software.

Jack's taxis

So let me introduce you to my friend Jack, who has decided to start his own taxi business. We’d better not push the analogies too far, but a not-too-serious look at the things Jack has to think about as he looks for a suitable vehicle and sets up his business operations might help us better understand some of the performance “metrics” we could be trying to optimize for OpenJ9.

1. Application startup time

First things first: how long will it take Jack’s new car to be ready to drive, from the moment he presses the ignition? Let’s say 5 seconds. Would Jack care if it were 10 seconds? Well, I probably wouldn’t, given the number of times I start my car in a day. But remember, Jack’s driving a taxi, stopping and starting all the time. He’d care!

The same goes for a truck driver starting his semi-trailer. (Call it a semi, 18-wheeler, transporter, big rig, artic… as you like!) Maybe that takes up to half a minute. But whether it’s actually 20 seconds or 30 probably wouldn’t matter too much if it’s just once in the morning. But for the mechanic who has to service that truck, frequently stopping and starting the engine to check progress, those 10 extra seconds add up.

At the other end of the size scale, what about Tom Cruise on his Ducati in Knight and Day? A 10 second delay for him – even once – would have been the difference between life and death!

It’s just the same for software applications. Does it matter? “It all depends…” The importance of startup time depends on the particular use case and business model.

Of course we can trim startup time by leaving some non-essential systems to start after the car is ready to drive. For example, it’s no problem if the audio system starts a few moments later.

Similarly, in Java application server software, we can have some initialization tasks still happening asynchronously in the background, after the server reports that it is ready to accept client requests.

OpenJ9 takes startup time seriously and has innovated in this area through the use of shared classes technology. By using a shared cache, and making common classes available to other JVMs, a great deal of time can be saved at startup for every subsequent JVM.

You can learn more about class sharing in these articles: Class sharing and Enhance performance with class sharing.

2. Application ramp-up time

But even after Jack’s started his car, and hit the freeway, it’ll take him a while to get up to maximum speed. No-one likes a car that is sluggish, especially when we have to quickly increase our speed to merge into traffic, or if we’re frequently stopping and starting. As a rule of thumb, the “ramp-up time” goes down as the horse power (and cost) goes up. But it’s something that Jack will have to balance.

Similarly, there are cases when it is absolutely essential that an application gets to peak throughput as soon as possible. For example, to handle a spike in traffic at your website so as to avoid losing business. Or if you’re using a framework such as Apache Hadoop or Apache Spark, which frequently launch multiple, short-lived instances of an application, perhaps.

At Eclipse OpenJ9 we have invested in complex JIT compilation heuristics and class loading capabilities that can be optimized to influence the speed with which an application ramps up.

You can learn more about JIT compilation in this article: Java performance optimizations.

3. Throughput

OK, we’ve sorted out the driving characteristics of Jack’s car by measuring startup and ramp-up times and making appropriate adjustments. But he’s in business to provide a taxi service; his main concern is to deliver as many customers as possible, as quickly as possible – in computing terms, his “throughput”.

The absolute maximum throughput would be if Jack could always drive at the speed limit, always have a full car, and travel the minimum distance to the destination. If any of those factors are not optimized, then his effectiveness – throughput – is reduced.

Software applications have similar issues. For maximum throughput, a large number of tasks must be completed as quickly as possible, and the JVM and application code must employ techniques that satisfy as many requests as possible in the shortest possible time – balancing the cost of, for example, JIT compilation, garbage collection, or thread pool management, against the ultimate benefit of running those tasks.

It might be like Jack spending a few moments stationary, optimizing the route in his Sat Nav, knowing that his overall journey time will be improved. Or maybe taking several customers on a single trip knowing that, although it might involve a slightly longer route for one, the total time taken to deliver everyone to their destinations is minimized.

We like to think that OpenJ9 achieves a good balance between different performance metrics, making sensible trade-offs to maximize throughput, but not at the expense of other measurements. But if you want to fine-tune it, OpenJ9 provides multiple tuning options so that you can optimize it for different workloads.

Next time:

In the next article, Jack needs to keep the customer satisfied (response time) at the same time as keeping his costs down (minimizing memory footprint). And if his business is successful, he’ll also have to consider how to accommodate even more customers (scalability).

 

Authors: Vijay Sundaresan and Peter Hayward

 

Footnote:
[1] www.snopes.com/humor/jokes/autos.asp

1 comment on"Measuring Java performance (or Jack starts a taxi business): Part 1"

  1. […] 本文翻译自:Measuring Java performance (or Jack starts a taxi business): Part 1(2017-10-25) […]

Join The Discussion

Your email address will not be published. Required fields are marked *