The Blog



Oracle Code One 2018 talk

Monday, October 22, 12:30p.m. Moscone West – Room 2024


The future is unknown, and the only constant in the software industry is change. Open projects often succeed by being creative, and creativity flourishes in a collaborative environment. At Eclipse OMR, Eclipse OpenJ9, and AdoptOpenJDK, we experience this first-hand every day. In test roles at these open projects, we must keep up with and often set the pace of change. We have gone through a rapid and occasionally tiring transformation in testing as we moved our projects and mission into the open.

This talk at Oracle Code One 2018 discusses our test transformation story, the challenges faced, and the principles/guidelines we used. We provide a brief overview of what areas we are testing and what tests we are utilizing. As curators of a fine collection of tests, the challenge is to make the different underlying test frameworks feel the same to developers.B By overlaying a homogenous look and feel, we give developers a common story to easily add, group, execute, and exclude tests.B This is accomplished by an intentionally simple framework to “bind together” the diverse set of underlying frameworks. This thin layer is called testkitgen. We developed it to quickly allow the incorporation of new suites of tests without the bloat of yet another full-blown test framework. The beauty of this approach is that it is based on (much!) feedback from developers.B As transparent and intuitive as possible, this thin veneer makes each of the major categories of tests we run (functional, openjdk, system, external, perf, and jck to date) feel similar, despite their great differences.

There is no hiding of what is running, no skirting away of variables or output, so developers familiar with one or more of the underlying frameworks get to see directly what is happening… no levels of indirection.B Developers with no experience ramp up quickly and find it easy to start running all types of tests, as the pattern to follow is predictable and easy.B We will walk through the framework with some examples and a quick demo. We will share our story of how we can quickly incorporate new tests and the on-going evolution of how we test languages. We will also discuss some techniques to measure success and strategies for collaboration with other teams that are developing new approaches to software verification.

If you find this interesting, please join us in the Oracle Code One session, or hop in and contribute in our Git repositories: