Imagine you finished the work on your DX application and you are ready to go into production. Now you want to release your awesome application as soon as possible. Since you are working for a well-known brand you know that your customers will expect a perfect user experience from your application. Therefore you review all the steps you’ve taken and you think to yourself:“We worked with the best design agency in our country to create an exceptional design. We even did some testing with future users of our application. For sure we thought about responsive design. We’ve also tested cross browser support for a great user experience. We also made sure to adopt test driven development in our development process. With this high degree of automated function verification tests we can be sure our application will work properly. I think we are ready for production.”
Isn’t that a good answer to the question if you are ready for production? Still there seems to be a big question mark. How did you ensure that the application will work as expected if thousands of users will work with the application at the same time? And how did you verify that no memory leaks are in your application? Sadly most of the time no effort is made from most projects to answer this question. Why?
You could argue that performance testing is cumbersome, complex, costly, and most of the time not needed at all because you don’t expect high loads of users. And furthermore you can always monitor your application in production and then tune your application based on the results made in the real world. This might be true for some use cases but as a rule of thumb you should think about performance testing if risk analysis indicates that failures caused by
- the response time of your application (speed)
- a non-adequately sized infrastructure for the load (capacity)
- missing capabilities for system growth (scalability)
- incorrect behavior under load (stability)
But why is it that such a crucial part of verification testing is missed out by so many teams? Gorakan Bjedov, a Senior Test Engineer at Google, described this in his blog entry with the presence of strong feelings in people about the topic: “feelings of fear (Oh, my God, I have no idea what to do because performance testing is so hard!), feelings of inadequacy (We bought this tool that does every aspect of performance testing, we paid so much for it, and we are not getting anything done!), feelings of confusion (So, what the heck am I supposed to be doing again?)”.
Here is my personal opinion: In most cases you don’t have to reach 100% accuracy in your performance tests to gain valuable insights into your application. It might not even be possible to reach this level if you think about more complex applications depending on thousands of parameters and external dependencies. Therefore it’s always to a good idea to get started with plausible assumptions and evaluate them over time with data from production. Because what’s really important to realize is that performance testing is not a one-off initiative. Instead you should start seeing it as a continuous part of your development cycle.
You still haven’t started implementing performance testing into your development process? Just go for it!
You already started thinking about performance tests but you wonder if there are some tips & trick for DX? Then this might be the right article for you to answer your questions.
- General Guidance for DX performance testing – https://developer.ibm.com/digexp/docs/docs/genperfguide/
- DX performance testing substitution patterns for reusable test suites – https://developer.ibm.com/digexp/docs/dxperftestsubstpatterns/
- Speed Matters – http://googleresearch.blogspot.co.uk/2009/06/speed-matters.html
- Powers of 10: Time Scales in User Experience – https://www.nngroup.com/articles/powers-of-10-time-scales-in-ux/
- Why milliseconds matter – https://www.dreamhost.com/blog/2013/03/14/website-speed-ftw-why-milliseconds-matter/
- Big data and analytics – http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=SA&subtype=ST&htmlfid=NIJ12345USEN