2021 Call for Code Awards: Live from New York, with SNL’s Colin Jost! Learn more

3 ways to write function tests for Liberty

Function tests describe what a system does and tests that a function of the system works as expected. Function testing is usually done by feeding a function a specific input and verifying the output matches what is expected. Since functional testing is a form of black-box testing, you can test the software’s functionality without knowing the internal parts of the application.

Three ways to function test with Liberty

There are many ways to do function testing with IBM WebSphere Liberty and Open Liberty. This article covers the 3 approaches that I think are best suited to those platforms:

  1. Using a build script/tool to start and stop the Liberty server
  2. Using a JUnit Rule to start and stop the Liberty server
  3. Using Arquillian to start and stop the Liberty server

Using a Maven build

Liberty supports the most common build tools such as Maven, Gradle, and Ant. You can use any of these tools to run tests against your applications, but this article focuses on Maven.

One benefit of building an application with a build system like Maven is that you can configure it to run a set of automated tests. The war plug-in extends the Java plug-in, which provides test tasks. You can write tests for the individual units of code outside of a running application server (unit tests), or you can write them to call the application that runs on the server (integration tests).

For a more in depth look at how to use Maven to run your functional tests, check out this guide: Testing the web application.

Advantages to using a common build tool

A few benefits to using a build tool like Maven, Ant, or Gradle include:

  • It keeps the Java code simple; there is no special setup code required in the JUnit test class.
  • The artifact you are testing is the user directory containing the config and the application WAR that was produced by the build so it is closely testing the production artifact.
  • It is straightforward to deploy the artifact remotely (for example, in the cloud) and run the test against that deployment without having to change your test code.

Disadvantages to using a common build tool

A few possible drawbacks to using common build tools include:

  • There’s an extra dependency on the Maven plug-in (or other build tool plug-in).
  • You can only automatically run the test from the build tool. You can’t it from Eclipse (or another IDE) without first manually starting the server.

Using a JUnit rule

Using a JUnit ClassRule allows you to use the Liberty System Programming Interfaces (SPI) to control your Liberty server from within your code. From here, you can configure your server while also being able to start and stop your server.

The example below is an example of when we programmatically started and stopped our Liberty server from within our test class:

    public class RuleFunctionalTest {

        public static ExternalResource serverResource = new ServerResource();

        public static class ServerResource extends ExternalResource {

            private final Server server;

            public ServerResource() {
                String usrDir = System.getProperty("liberty.usr.dir");
                String serverName = System.getProperty("liberty.server.name");
                ServerBuilder sb = new ServerBuilder();
                server = sb.setName(serverName).setUserDir(new File(usrDir)).build();

            protected void before() throws Throwable {
                Future<Result> startReturnCode = server.start();
                Result result = startReturnCode.get();
                assertEquals(true, result.successful());

            protected void after() {
                Future<Result> stopReturnCode = server.stop();
                try {
                    Result result = stopReturnCode.get();
                    assertEquals(true, result.successful());
                } catch (InterruptedException | ExecutionException e) {
                    fail("Caught exception stopping server" + e.getMessage());

Advantages to using a JUnit rule

A few benefits to using a JUnit rule include:

  • Using a JUnit rule keeps the build code simple. You simply create a standard Maven setup of a new test task and sourceSet.
  • You can run it using an IDE as well as through the build.
  • There are no external dependencies.

Disadvantages to using a JUnit rule

A few drawbacks to using a JUnit rule include:

  • Using this rule complicates the Java code because it is responsible for starting and stopping the Liberty server as well as for running the tests.
  • Deploying to different locations is harder because it is all done through the Java API.
  • You are required to build a fully functional Liberty user directory structure, including your server configuration and application. In this example, this suits the situation quite well because that is already the output from the build, but that would not always be the case.

Using Arquillian

The final technique for doing function tests in Libery uses Arquillian, a testing framework to develop automated functional, integration, and acceptance tests for your Java applications. Arquillian sets up the test environment and handles the application server lifecycle for you so you can focus on writing tests.

If you want a more in-depth look at how to use Arquillian with Liberty, check out this guide on the openliberty.io site: Testing microservices with Arquillian managed container.

Advantages to using Arquillian for function testing

A few benefits of using Arquillian include:

  • It allows you to create and deploy a test archive through the test using ShrinkWrap. There is no need to build a fully functional Liberty server prior to running the test.
  • You can run Arquillian using an IDE as well as through the build.
  • Arquillian is the most feature-rich option. For instance, you can run your tests against multiple containers (application servers) and remote servers.
  • The correct URL for the application is passed to the tests, so it is more flexible for controlling the port numbers, and so on, of your test environment.

Disadvantages to using Arquillian for function testing

  • Arquillian still needs to have a skeleton server defined and it has to go into the main server runtime installation location (there is no support for a separate user directory).
  • The application you are testing is one created by Arquillian through the ShrinkWrap tool. You won’t deploy this actual application or have this server configuration.


If you are already building a Liberty user directory, then the simplicity of using either the build tool or a JUnit Rule to start and stop your server is appealing. If, however, you want the ability to control exactly what is deployed, or you want to test against multiple containers, then Arquillian offers these features.