I recently wrote some functional/integration/system tests (depending on your naming preference!) for some of our samples on Github. The tests that I was writing would connect to an application on a running Liberty server. As always, there were lots of different ways to do this; I chose the one that best fitted with our environment for our samples.

The other techniques were interesting too, though, so I’ll cover three of them in this post:

  • Using the build script to start and stop the Liberty server
  • Using a JUnit Rule to start and stop the server
  • Using Arquillian to start and stop the server

I’ve provided an example in GitHub that provides both a sample application, and the tools or scripts required to try all three approaches to test it. Before I dig into the details of each technique I’ll give a short explanation of the sample structure.

The functional tests sample

The sample on GitHub is made up of three projects: a root project (sample.functionaltest), a project for the application code (functionaltest-application) and a project for the Liberty configuration (functionaltest-wlpcfg). The projects are built using Gradle and when this happens the application WAR file is packaged and placed into a server in the wlpcfg project. The wlpcfg project can then be used as a user directory in Liberty and WDT in Eclipse.

The application itself is a very simple “Hello, World!” servlet. As with a real world application, the servlet class itself (Greeting.java) is just a very thin layer that then calls down to GreetingGenerator.java which is the actual business logic (and yes, I did just describe something that prints “Hello, World!” as business logic!). This makes the business logic easier to test and does not rely on any of the Liberty or Java EE APIs.

The sample includes a simple JUnit test of GreetingGenerator. Whether or not the Greeting servlet itself (as a thin wrapper) is unit tested will depend on your preference. For this article, we are more interested in how integration tests work, so that is where our focus will be. We want to know that the end-to-end solution deployed to Liberty actually works. The fast start-up speed of Liberty ensures this does not hit our build times too hard.

Each testing technique has its own Gradle build file and sourceSet to make it clear which is doing what. All of these files are contained within the functionaltest-application project.

The overall project structure is:

  +- build.gradle                              -- build file for building all projects
  +- functionaltest-application                -- project for the application code
    +- build.gradle                            -- build file that creates the WAR, runs the unit tests and imports the other gradle build files
    +- gradle.properties                       -- properties file defining where Liberty is installed
    +- fvtBuild.gradle                         -- build file for FVT using the build to start and stop the server
    +- fvtRule.gradle                          -- build file for FVT using a JUnit rule to start and stop the server
    +- fvtArquillian.gradle                    -- build file for FVT using Arquillian to deploy the application to Liberty and start and stop the server
    +- src
      +- main                                  -- sourceSet for application code
      +- test                                  -- sourceSet for unit tests
      +- fvtBuild                              -- sourceSet for FVT using the build to start and stop the server
      +- fvtRule                               -- sourceSet for FVT using a JUnit rule to start and stop the server
      +- fvtArquillian                         -- sourceSet for FVT using Arquillian to deploy the application to Liberty and start and stop the server
  +-functionaltest-wlpcfg                      -- project for server configuration
    +- servers
      +- functionalTestSample                  -- specific server configuration
        +- server.xml                          -- server configuration
        +- apps                                -- directory for applications (generated by build)
          +- async-jaxrs-application.war       -- sample application (generated by build)

FVT techniques

Below is a short description of the key files used to make each technique work, for the complete solution see the sample on GitHub.

Using the build

There is a Liberty Gradle plug-in which enables you to write build scripts to start and stop the server. (There are also Maven and Ant plug-ins that achieve the same thing.) In the fvtBuild.gradle file, a new fvtBuild task is added to the build lifecycle that uses the Liberty plug-in to start and stop the server before and after running the BuildFunctionalTest JUnit test:

// Create a task to actually run the tests
task fvtBuild(type: Test, dependsOn: jar) {
    group 'Verification'
    description 'Runs the functional verification tests by using Gradle to start liberty server prior to running all the tests and then stopping the server afterwards.'
    testClassesDir = sourceSets.fvtBuild.output.classesDir
    classpath = sourceSets.fvtBuild.runtimeClasspath
    dependsOn publishWar, libertyStart
    finalizedBy libertyStop
    reports.html.destination = file("$buildDir/reports/fvtBuild")
    reports.junitXml.destination = file("$buildDir/test-results/fvtBuild")

check.dependsOn fvtBuild

// Set the ordering so we only start the server after the WAR has been published
libertyStart.mustRunAfter publishWar

// Tell the liberty plugin about the server to use in the test
liberty {
    wlpDir = "${libertyRoot}"
    serverName = 'functionalTestSample'
    userDir = '../functionaltest-wlpcfg'
    clean = true


  • It keeps the Java code simple; there is no special setup code required in the JUnit test class.
  • The artifact you are testing is the user directory containing the config and the application WAR that was produced by the build so it is closely testing the production artifact.
  • It is straightforward to deploy the artifact remotely (e.g. to IBM Bluemix) and run the test against that deployment without having to change your test code.


  • There’s an extra dependency on the Gradle plug-in.
  • It requires a fully-functional Liberty user directory structure, including both the server configuration and the application. In this example, this is already the output from the build, but that may not always be the case.
  • You can only automatically run the test from the build script; you could not run it from Eclipse (or other IDE) without first manually starting the server.

Using a JUnit Rule

Here the fvtRule.gradle files create a new fvtRule task to run the RuleFunctionalTest JUnit test. All of the server starting and stopping is handled by a JUnit ClassRule within that class which uses the Liberty SPI to control the server:

public class RuleFunctionalTest {
     * The ExternalRule will start and stop the server for us
    public static ExternalResource serverResource = new ServerResource();

     * <p>This class will construct a {@link Server} instance based on two system properties being set:</p>
     * <ul>
     * <li>liberty.usr.dir: The user directory containing the server to start</li> 
     * <li>liberty.server.name: The name of the server to start</li> 
     * </ul>
     * @see http://www-01.ibm.com/support/knowledgecenter/SSAW57_8.5.5/com.ibm.websphere.wlp.nd.multiplatform.doc/ae/twlp_extend_embed.html?lang=en
    public static class ServerResource extends ExternalResource {

        private final Server server;

        public ServerResource() {
            String usrDir = System.getProperty("liberty.usr.dir");
            String serverName = System.getProperty("liberty.server.name");
            ServerBuilder sb = new ServerBuilder();
            server = sb.setName(serverName).setUserDir(new File(usrDir)).build();
        protected void before() throws Throwable {
            Future<Result> startReturnCode = server.start();
            Result result = startReturnCode.get();
            assertEquals(true, result.successful());
        protected void after() {
            Future<Result> stopReturnCode = server.stop();
            try {
                Result result = stopReturnCode.get();
                assertEquals(true, result.successful());
            } catch (InterruptedException | ExecutionException e) {
                fail("Caught exception stopping server" + e.getMessage());


    // ...Tests...


  • It keeps the build code simple; it is just a standard Gradle setup of a new test task and sourceSet.
  • It can be run using Eclipse as well as through the build.
  • There are no external dependencies.


  • It complicates the Java code because it is responsible for starting and stopping the Liberty server as well as for running the tests.
  • It is harder to deploy to different locations because it is all done through the Java API.
  • It requires you to build a fully-functional Liberty user directory structure, including your server configuration and application. In this example, this suits the situation quite well because that is already the output from the build, but that would not always be the case.

Using Arquillian

The final technique uses Arquillian to control the server as Nichole describes in
Getting started with Liberty and Arquillian
. The arquillianFvt.gradle file sets up a Liberty server in the liberty installation. It also installs Arquillian and defines the arquillianFvt task to run the ArquillianFunctionalTest JUnit test. The arquillian.xml file defines the Liberty server to use so that the ArquillianFunctionalTest JUnit test can use the Arquillian APIs to deploy an application and start and stop the server:

public class ArquillianFunctionalTest {

    public static WebArchive createDeployment() {
        return ShrinkWrap.create(WebArchive.class, "test.war").addClasses(Greeting.class, GreetingGenerator.class);

    public void testServletWithName(@ArquillianResource URL url) throws IOException {
        URL testUrl = new URL(url, "Hello?name=Iain");
        String response = getResponse(testUrl);
        assertEquals("Hello, Iain!", response);

    public void testServletWithDefaultMessage(@ArquillianResource URL url) throws IOException {
        URL testUrl = new URL(url, "Hello");
        String response = getResponse(testUrl);
        assertEquals("Hello, World!", response);

    // ...utilities...


  • It allows you to create and deploy a test archive through the test using ShrinkWrap. There is no need to build a fully-functional Liberty server prior to running the test.
  • It can be run using Eclipse as well as through the build.
  • This is the most feature-rich option; for instance, you can run your tests against multiple containers (application servers) and remote servers.
  • The correct URL for the application is passed to the tests so it is more flexible for controlling the port numbers, and so on, of your test environment.


  • The setup is spread across more locations. You have to add the Arquillian setup to your build, write your JUnit specifically to set up the test archive, and have an arquillian.xml file for configuring the server.
  • It still need to have a skeleton server defined and it has to go into the main server runtime installation location (there is no support for a separate user directory).
  • The application you are testing is one created by Arquillian through the ShrinkWrap tool. It is not what you will actually be deploying and neither is the server configuration.


If you are already building a Liberty user directory then the simplicity of using either the build or a JUnit Rule to start and stop your server is appealing. If, however, you want the ability to control exactly what is deployed, or you want to test against multiple containers, then Arquillian offers these features.

I hope you found this useful, feedback is welcome!

See Also

Unit Testing for Java EE (Adam Bien, Oracle)
Integration Testing for Java EE (Adam Bien, Oracle)

1 comment on"Writing functional tests for Liberty"

  1. Hi Ian,

    For the arquillian approach, would it make sense to use https://github.com/WASdev/ci.gradle (liberty gradle plugin) and download liberty from the maven central repo? Do you have any plans for enhancing these samples with that approach? This would work great in a CI environment (e.g., Jenkins) where by simply updating the version number in a gradle file, the test suite could be run on a “new” version of Liberty without any explicit setup.

    Cheers and thanks in advance.

Join The Discussion

Your email address will not be published. Required fields are marked *