Startup time, ramp-up time, and throughput
Youāve probably heard those jokes that start, “If my car behaved like softwareā¦ā ā including that infamous exchange between Bill Gates and General Motors twenty years ago. Itās an urban myth, of course, but fun nevertheless :
Gates: āIf GM had kept up with technology like the computer industry has, we would all be driving $25 cars that get 1,000 miles to the gallon.ā
GM: āAnd if General Motors had developed technology like Microsoft, we would all be driving cars that crash twice a day.ā
And that retort was followed by a list of other unfortunate ācharacteristicsā of this thoroughly modern vehicle, including a reminder to press the Start button to shut off the engine when youāre done!
Well, maybe the parallel between computers and cars is best avoided.
Or maybe we can give it another try to help us think about the performance of Java: in particular, the Eclipse OpenJ9 Java Virtual Machine (JVM) technology used for running Java programs.
Of course, āperformanceā means different things to different people and gets measured in different ways. Interestingly, the same complications seem to apply as much to cars as to software.
So let me introduce you to my friend Jack, who has decided to start his own taxi business. Weād better not push the analogies too far, but a not-too-serious look at the things Jack has to think about as he looks for a suitable vehicle and sets up his business operations might help us better understand some of the performance āmetricsā we could be trying to optimize for OpenJ9.
1. Application startup time
First things first: how long will it take Jackās new car to be ready to drive, from the moment he presses the ignition? Letās say 5 seconds. Would Jack care if it were 10 seconds? Well, I probably wouldnāt, given the number of times I start my car in a day. But remember, Jackās driving a taxi, stopping and starting all the time. Heād care!
The same goes for a truck driver starting his semi-trailer. (Call it a semi, 18-wheeler, transporter, big rig, articā¦ as you like!) Maybe that takes up to half a minute. But whether itās actually 20 seconds or 30 probably wouldnāt matter too much if itās just once in the morning. But for the mechanic who has to service that truck, frequently stopping and starting the engine to check progress, those 10 extra seconds add up.
At the other end of the size scale, what about Tom Cruise on his Ducati in Knight and Day? A 10 second delay for him ā even once ā would have been the difference between life and death!
Itās just the same for software applications. Does it matter? āIt all dependsā¦ā The importance of startup time depends on the particular use case and business model.
Of course we can trim startup time by leaving some non-essential systems to start after the car is ready to drive. For example, itās no problem if the audio system starts a few moments later.
Similarly, in Java application server software, we can have some initialization tasks still happening asynchronously in the background, after the server reports that it is ready to accept client requests.
OpenJ9 takes startup time seriously and has innovated in this area through the use of shared classes technology. By using a shared cache, and making common classes available to other JVMs, a great deal of time can be saved at startup for every subsequent JVM.
2. Application ramp-up time
But even after Jackās started his car, and hit the freeway, itāll take him a while to get up to maximum speed. No-one likes a car that is sluggish, especially when we have to quickly increase our speed to merge into traffic, or if weāre frequently stopping and starting. As a rule of thumb, the āramp-up timeā goes down as the horse power (and cost) goes up. But itās something that Jack will have to balance.
Similarly, there are cases when it is absolutely essential that an application gets to peak throughput as soon as possible. For example, to handle a spike in traffic at your website so as to avoid losing business. Or if youāre using a framework such as Apache Hadoop or Apache Spark, which frequently launch multiple, short-lived instances of an application, perhaps.
At Eclipse OpenJ9 we have invested in complex JIT compilation heuristics and class loading capabilities that can be optimized to influence the speed with which an application ramps up.
You can learn more about JIT compilation in this article: Java performance optimizations.
OK, weāve sorted out the driving characteristics of Jackās car by measuring startup and ramp-up times and making appropriate adjustments. But heās in business to provide a taxi service; his main concern is to deliver as many customers as possible, as quickly as possible ā in computing terms, his āthroughputā.
The absolute maximum throughput would be if Jack could always drive at the speed limit, always have a full car, and travel the minimum distance to the destination. If any of those factors are not optimized, then his effectiveness ā throughput ā is reduced.
Software applications have similar issues. For maximum throughput, a large number of tasks must be completed as quickly as possible, and the JVM and application code must employ techniques that satisfy as many requests as possible in the shortest possible time ā balancing the cost of, for example, JIT compilation, garbage collection, or thread pool management, against the ultimate benefit of running those tasks.
It might be like Jack spending a few moments stationary, optimizing the route in his Sat Nav, knowing that his overall journey time will be improved. Or maybe taking several customers on a single trip knowing that, although it might involve a slightly longer route for one, the total time taken to deliver everyone to their destinations is minimized.
We like to think that OpenJ9 achieves a good balance between different performance metrics, making sensible trade-offs to maximize throughput, but not at the expense of other measurements. But if you want to fine-tune it, OpenJ9 provides multiple tuning options so that you can optimize it for different workloads.
In the next article, Jack needs to keep the customer satisfied (response time) at the same time as keeping his costs down (minimizing memory footprint). And if his business is successful, he’ll also have to consider how to accommodate even more customers (scalability).
Authors: Vijay Sundaresan and Peter Hayward