In Part 1 of this series, you migrated the Daytrader3 application from IBM® WebSphere® Application Server Liberty 8.5.5.0 to the latest version of Liberty server. Then, in Part 2, you migrated the Daytrader3 application that is running in an on-premises Liberty 17.0.0.2 server to the cloud platforms of IBM Cloud Private (with Docker and Kubernetes) and IBM Cloud Public (with Cloud Foundry). In Part 3, you added build automation to the monolith by using the Maven industry standard build lifecycle and augmented the lifecycle with plug-in bindings to support continuous integration of the monolith. Next, in Part 4, you configured Jenkins for build automation, continuous integration, and continuous delivery. Finally, this tutorial will teach you how to refactor the monolithic source of the Daytrader3 application to microservices that you can separately manage, deploy, and scale.

To continue with this part, you must have the workspace, WebSphere Liberty server, and Maven configuration from Parts 1 – 4. If you did not complete the steps in these parts, download Part4Finish.zip from the GitHub repository.

Prerequisites

Overview of refactoring

The following figure highlights the key architectural differences before microservices refactoring.

Daytrader3 application before microservices refactoring

After refactoring, the monolithic Daytrader3 application is divided into several microservices according to the business functions that they provide. The following figure highlights the key architecture differences after the microservice refactoring. These differences include moving the web project into its own stand-alone project, refactoring the Enterprise JavaBeans (EJB) module into multiple REST web modules, and connecting all of the REST web modules to a stand-alone Postgre database.

Daytrader3 application after microservices refactoring

The Daytrader3 application was implemented as a demonstration application for WebSphere Application Server. It includes many features and functions that showcase the server features. This part, Part 5 in this series, focuses on refactoring codes that provide the trading and account management features of the Daytrader3 application. These features represent implementations that reflect real-life business use cases.

Reconstruct the Rest project module

The Rest project module contains a set of address-related services that does not have any dependencies on the other modules of the Daytrader3 application. It also exposes a REST API so that its dependents can start it after it is converted to a microservice, making the Rest project module a good choice as a first step in refactoring the monolith.

At a high level, the following figure shows the existing packaging structure and invocation pattern of the Rest module in relation to the Daytrader enterprise application before and after refactoring.

Rest module before and after refactoring

The following steps guide you through the required changes to split the Rest project module into an independent microservice project in the Eclipse workspace.

Update the pom.xml file

To begin, update the pom.xml file of the module to compile, package, and test the Rest code. For this example, you make the following changes to the pom.xml file of the Rest module:

  • Because you are compiling the module independently, you update the <dependencies> element under the root <project> element of the pom.xml file to include support for JAX-RS, Java Enterprise Edition (Java EE) Web Profile, and JUnit. The following dependencies show the result:
<dependency>
      <groupId>org.glassfish.jersey.core</groupId>
      <artifactId>jersey-client</artifactId>
      <version>2.22.2</version>
      <scope>compile</scope>
</dependency>
<dependency>
      <groupId>javax</groupId>
      <artifactId>javaee-web-api</artifactId>
      <version>6.0</version>
      <scope>provided</scope>
</dependency>
<dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
</dependency>
  • To simulate the current testing procedure that is used for the monolith application, you copy the <profiles> element from the pom.xml file of the daytrader-ee6 module. The following updates, among others, are needed:

    • Remove the redundant dependency sections.
    • Reduce the Liberty Maven Plug-in features.
    • Update the deployable packing names
    • Update the Docker image names.
    • Update the host and context path information.

Create integration tests

Previously, integration tests were run from the enterprise archive (EAR) module. This method is no longer possible for the Rest module because you are separating it from the Daytrader monolith. Therefore, you must develop a new set of test cases for the Rest services. For an example of test case implementations, review the following RestIT.java code listing.

package com.ibm.websphere.samples.daytrader.rest;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Invocation;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.Response;

import org.junit.Test;
import static org.junit.Assert.assertTrue;

public class RestIT {

       @Test
       public void testGetAddresses() {
               testEndpoint("/rest/addresses", "GET");
       }

       @Test
       public void testGetAddress() {
               testEndpoint("/rest/addresses/Entry4", "GET");
       }

       @Test
       public void testSearch() {
               testEndpoint("/rest/addresses/search/Entry4", "GET");
       }

       private void testEndpoint(String endpoint, String httpMethod) {
               String route = System.getProperty("daytrader.app.route");

               Response response = sendRequest(route + endpoint, httpMethod);
               int responseCode = response.getStatus();
               assertTrue("Incorrect response code: " + responseCode, responseCode == 200);
               String responseString = response.readEntity(String.class);
               System..println(responseString);
               response.close();
       }

       private Response sendRequest(String url, String method) {
               Client client = ClientBuilder.newClient();
               WebTarget target = client.target(url);
               Invocation.Builder invoBuild = target.request();
               Response response = response = invoBuild.build(method).invoke();
               return response;
       }
}

Configure the WebSphere Liberty server

To run the Rest services so that you can test them independently, configure a WebSphere Liberty server for the Rest project. For simplicity, reuse the Liberty server that you configured for the Daytrader EAR module.

  1. Copy the /src/main/liberty and /src/main/resources folders in the daytrader-ee6 project to the Rest project.
  2. Reduce the configured Liberty features just as you did in the pom.xml file of the Liberty Maven Plug-in.
  3. Remove the Java Database Connectivity (JDBC) and Java Message Service (JMS)-related configurations.

The following listing shows the resulting server.xml file.

            <server>
                  <featureManager>
                        <acceptLicense>true</acceptLicense>
                        <feature>localconnector-1.0</feature>
                        <feature>jaxrs-1.1</feature>
                        <feature>jndi-1.0</feature>
                  </featureManager>

                  <httpEndpoint host="*" httpPort="9080" id="defaultHttpEndpoint"/>

            </server>

Create a webapp deployment descriptor

When you make the Rest module independent, the module loses the context root configurations that are provided in the EAR module. To resolve this problem, add your own webapp deployment descriptor. For this example, you create a new /src/main/webapp/WEB-INF/ibm-web-ext.xml file with the following contents.

<?xml version="1.0" encoding="UTF-8"?>
<web-ext xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
      xmlns="http://websphere.ibm.com/xml/ns/javaee"
      xsi:schemaLocation="http://websphere.ibm.com/xml/ns/javaee http://websphere.ibm.com/xml/ns/javaee/ibm-web-ext_1_0.xsd"
      version="1.0">
      <context-root uri= />
</web-ext>

Verify the Rest services

You are now ready to build, package, deploy, and test the Rest project. Run the Maven verify goal with the ci-liberty profile or ci-docker profile to verify the Rest project services.

For the command line, go to the Rest module folder and run the following Maven command:

  • For the test with Liberty, enter:

    mvn verify -Pci-liberty

  • For the test with Docker, enter:

    mvn verify -Pci-docker

For Eclipse, create and run the Maven Build launch configuration for the ci-liberty or ci-docker profile. For ci-liberty, in the Create, manage, and run configurations window, complete the following steps:

  1. In the left navigation pane, click Rest verify liberty.
  2. In the Name field, enter Rest verify liberty.
  3. In the Goals field, enter verify.
  4. In the Profiles field, enter ci-liberty.
  5. Click Run.

    Test with Liberty

For ci-docker, in the Create, manage, and run configurations window, complete the following steps:

  1. In the left navigation pane, click Rest verify liberty.
  2. In the Name field, enter Rest verify docker.
  3. In the Goals field, enter verify.
  4. In the Profiles field, enter ci-docker.
  5. Click Run.

    Test with Docker

If your initial testing of the independent Rest application module generates errors, correct the incomplete coding in the com.ibm.websphere.samples.daytrader.rest.AddressBook class. In our testing, the error resulted in the following code.

@Path("/{entryName}")
public Address getAddress(@PathParam(value = "entryName") String entryName) {
       Address addr = AddressBookDatabase.getAddress(entryName);
       return addr;
}

We made the following changes to correct the code.

@GET
@Path("/{entryName}")
@Produces({ "application/json" })
public Address getAddress(@PathParam(value = "entryName") String entryName) {
       Address addr = AddressBookDatabase.getAddress(entryName);
       return addr;
}

After you correct the code, the log file and failure trace report indicate that the build was successful.

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.055 s
[INFO] Finished at: 2017-08-24T16:59:10-07:00
[INFO] Final Memory: 14M/26M
[INFO] ------------------------------------------------------------------------

Failure trace report showing a successful build

Clean up

The Rest service project is now an independent microservice. It is self-contained, with no dependencies on any other projects in the Daytrader3 workspace. You can also deploy and scale it independently.

As a final step in reconstructing the Rest project, clean up its associations with other projects in the Daytrader monolith workspace. You must modify the modules that currently depend on the services that the Rest provides to access the independent Rest services. We explain these modifications later in this tutorial in the steps to reconstruct those modules.

For now, remove Rest from the Daytrader EAR project. That is, update and remove the <dependencies> and <plugin> elements that are related to the Rest module in the pom.xml file for the daytrader-ee6 EAR module.

You have now completed refactoring of the Rest service module in the Daytrader3 monolith application to an independent microservice project.

Reconstruct the dt-ejb project module

The dt-ejb project module contains the core business functions of the Daytrader3 application. It consists of implementations that are related to trading activities and account management. For this example, these functions are concurrently implemented by using EJB and direct JDBC-based objects (plain old Java objects (POJOs)). It also provides implementations for configurations and database management features that are purely for demonstration purposes.

In refactoring this module, you refactor the direct JDBC POJO implementations of the core business functions because they are more aligned with common microservices development approaches and are much less container dependent. This section highlights the steps to refactor the dt-ejb project module into microservices.

Analyze the dt-ejb project functions to extract microservices

The goal of refactoring the dt-ejb project is to separate the business logic implementations from the web-tier codebase. You want to remove the direct Java dependencies that the web project has on the dt-ejb project. This approach enables the component-based development of the two tiers with communication via loosely coupled RESTful protocols. You also want to break up the dt-ejb code base according to the business functions that they provide. The resulting microservices components allow for more agile development management, more flexible deployment architecture, and more robust and targeted workload scaling strategy.

When you analyze the dt-ejb implementations, most of the functional calls from the web tier to the business logic tier go through the TradeAction.java method. Therefore, it is a key class to evaluate when you need to determine how to separate the two tiers and how to break up the dt-ejb logic itself. Close examination reveals that the TradeAction.java method provides public methods for all business operations that are exposed by the dt-ejb module. In fact, the TradeAction.java method acts as a gateway dispatcher for dt-ejb, and these public methods are a set of APIs for the core business logic that dt-ejb provides. If you divide these API methods into functional groups and expose them as RESTful interface methods, you can separate the web and business logic tiers, and split the module into smaller microservices. The result from our analysis is four microservices that each represent the business functions of Market, Accounts, Quotes, and Settings (database settings).

The following table lists the grouping of the business operations and their designated RESTful API URI and HTTP methods. Many other approaches are possible for you to design the functional capabilities of your microservices and your RESTful APIs, but they are not addressed in this article. Choose the approach that best suits the needs of your organization or architecture.

Table 1.

Business Operation (TradeAction.java method) URI HTTP

Market | getMarketSummary() | /markets/ | GET | Accounts | getHoldings(userID) | /accounts/{userID}/holdings | GET | | getAccountData(userID) | /accounts/{userID} | GET | | getAccountProfileData(userID) | /accounts/{userID}/profile | GET| | updateAccountProfile(accountProfileData) | /accounts/{userID}/profile/{password}/{fullName}/{address}/{emails}/{creditCard}/ | PUT | | login(userID, password) | /accounts/{userID}/{password} | PUT | | logout(userID) | /accounts/{userID} | PUT | | register(userID, password, fullname, address, email, creditCard, openBalance) | /accounts/{userID}/{password}/{fullName}/{address}/{emails}/{creditCard}/{openBalance} | POST | | buy(userID, symbol, quantity, orderProcessingMode) | /accounts/{userID}/buyorders/{symbol}/{quantity}/orderProcessingMode} | POST | | sell(userID, holdingID, orderProcessingMode) | /accounts/{userID}/sellorders/{holdingID}/{orderProcessingMode} | POST | | getOrders(userID) | /accounts/{userID}/orders | GET | | getClosedOrders(userID) | /accounts/{userID}/closedorders | PUT | Quotes | createQuote(symbol, companyName, price) | /quotes/{symbol}/{companyName}/{price} | POST | | getQuote(symbol) | /quotes/{symbol} | GET | | getAllQuotes() | /quotes/ | GET | | updateQuotePriceVolume(symbol, changeFactor, sharesTraded) | /quotes/{symbol}/{chageFactor}/(sharesTraded) | PUT | | Settings | | resetTrade(deleteAll) | /settings/{deleteAll} | DELETE | | recreateDBTables() | /settings/tables | PUT | | buildDB() | /settings/databases | PUT|

* These methods were not part of the interface but were added to externalize database.

You now know which microservices you want to create. The following steps explain how to move and refactor the existing dt-ejb codes into their own microservices projects. The following figure highlights the key components of the dt-ejb module before you complete the refactoring steps in this tutorial.

The dt-ejb module before microservices refactoring

The following figure highlights the key components for Market, Accounts, Quotes, and Settings of the dt-ejb module after you complete the refactoring steps.

The dt-ejb module after microservices refactoring

Refactor existing implementations to create a microservices module

Implement the microservices from the table in Step 2a. Because you already determined the URL structure and how you want to split up the workload, the implementation of these RESTful services is easier for this step.

Before you start moving and changing codes, you must understand that the various business logic implementations within the dt-ejb module depend on a set of model objects and interfaces. Because you will divide the business logic codes into separate modules, first, remove these dependencies and place them in a library project. This way, each of the new microservices modules can reference them.

For this step, create a dt-ejb-client Java project in Eclipse and move the following common dependencies into it:

  • com.ibm.websphere.samples.daytrader.*
  • AccountDataBean.java
  • AccountProfileDataBean.java
  • HoldingDataBean.java
  • MarketSummaryDataBean.java
  • OrderDataBean.java
  • QuoteDataBean.java
  • RunStatsDataBean.java
  • TradeConfig.java
  • TradeServices.java
  • com.ibm.websphere.samples.daytrader.util.*
  • FinancialUtils.java
  • Log.java

Now that you have proper references to the common dependencies, create a new microservice by using the Rest project that was reconstructed in Step 1 as a base. Then, apply refactored dt-ejb business logic implementations on top by using the following steps:

  1. Copy the Rest project in Eclipse.
  2. Rename the project to marketsMicroService.
  3. Update the artifactID and name in the pom.xml file to marketsMicroService.
  4. Add a dependency to dt-ejb-client.jar in the pom.xml file.
  5. Create a RESTful service and application classes: Markets.java and MarketsApplication.java.
  6. Copy the code for business operations in the TradeAction.java method for the Markets functions (see the table in Step 2a) into Markets.java (for example: TradeAction. _getMarketSummary()).
  7. Add annotations to expose the business operations in the RESTful service as shown in the following listing.
       /**
       * @see TradeServices#getMarketSummary()
       */
       @GET
       @Path("/markets")
       @Produces({"application/json"})
       public MarketSummaryDataBean getMarketSummary() throws Exception {
             /** existing code from TradeAction.java **/
}

Externalize the database to allow the microservices to run on their own server

For simplicity and portability, the Daytrader3 application uses a Derby database that is built and created in memory. That is, the database exists within the Java virtual machine (JVM) and is accessible only by the components of the Daytrader3 application that run within the same JVM. This internal location of the database is not compatible with our microservices because each microservice will be in its own JVM.

Therefore, externalize the database so that all microservices can remotely access it. The Daytrader3 application supports externalized databases, such as DB2, Oracle, and Derby. It can programmatically determine which database you are using and loads the database-specific Data Definition Language (DDL) file. In this tutorial, we use a PostgreSQL database with added support so that later we can use the free-trial offerings from ElephantSQL, which is a PostgreSQL database service on the IBM Cloud Public platform.

To externalize the database:

  1. Create and start a PostgresSQL database locally with Docker:

    docker run --name dayTraderPostgres –p 5432:5432 –e POSTGRES_PASSWORD=mysecretpassword -d postgres
    
    docker start dayTraderPostgres
    
  2. Modify the Liberty server configuration. Replace the JDBC and data source configuration with the code that is shown in the following listing. Add the PostgreSQL JAR files to the library directory.

         <jdbcDriver id="postgresqlJDBC">
              <library name="Postgresql Libs">
    <fileset dir="${shared.resource.dir}/Daytrader3SamplePostgresqlLibs" id="Postgresql Jars" includes="*.jar"/>
              </library>
         </jdbcDriver>
    
         <dataSource connectionManagerRef="conMgr1" id="TradeDataSource" isolationLevel="TRANSACTION_READ_COMMITTED" jdbcDriverRef="postgresqlJDBC" jndiName="jdbc/TradeDataSource" statementCacheSize="60" type="javax.sql.DataSource">
              <properties databaseName="postgres" password="mysecretpassword" portNumber="5432" serverName="localhost" user="postgres"/>
         </dataSource>
    
         <dataSource connectionManagerRef="conMgr2" id="NoTxTradeDataSource" isolationLevel="TRANSACTION_READ_COMMITTED" jdbcDriverRef="postgresqlJDBC" jndiName="jdbc/NoTxTradeDataSource" statementCacheSize="10" transactional="false">
              <properties databaseName="postgres" password="mysecretpassword" portNumber="5432" serverName="localhost" user="postgres"/>
         </dataSource>
    
  3. Modify the code to use the PostgreSQL database. Because PostgreSQL does not have a DDL file, modify the existing DDL from another database to work with Postgres. The final DDL file is in the completed workspace:

    <wlp>/usr/server/daytraderSettings/dbscripts/postgresql/Table.ddl
    

Test and verify the microservices

Deploy the marketsMicroService independent of the Daytrader3 application, and make REST calls to the microservice by using the URI that is specified in the table in Step 2a. The full URL looks similar to this example:

http://localhost:9080/marketsMicroService/markets

The following figure shows an example of deployment in a web browser.

Invoke marketsMicroService from a web browser

You can test the RESTful service manually by using a SOAPUI, for example. Ideally, you must create test cases to run during the Maven verify stage. The following example shows a test case for the microservice.

@Test
public void testGetMarketSummary() throws Exception {
      String responseString = invokeEndpoint(marketsRoute + "/markets/", "GET");
      MarketSummaryDataBean responseObject;
      responseObject = mapper.readValue(responseString,MarketSummaryDataBean.class);
      Assert.assertNotNull(responseObject);
}

The invokeEndpoint() method is a helper method to clean the code. You can inspect the code in the solution for effective ways to make REST calls. The pom.xml file might also require updates to ensure that test cases are run.

Complete the refactoring and clean ups

You have completed the refactoring of the dt-ejb module to create the marketsMicroService. You can repeat Steps 2b and 2d to further refactor dt-ejb to create Accounts, Quotes, and Settings microservices. The database externalization is for all migration. You do not need to repeat it for each microservice refactoring.

Reconstruct the web project module

The web module is the last project module that you need to reconstruct and extract from the Daytrader3 application. As a front-end UI component, the web module has dependencies on the Rest and dt-ejb modules. Communications with the Rest module were performed by using a RESTful-style API. The communication channel has not changed after the Rest module was reconstructed. Exchanges with the dt-ejb module were done by using Java style APIs that are no longer available since dt-ejb was refactored to RESTful microservices. That is, the web module must be refactored to leverage the new microservices APIs.

The following figure shows the state of the web project module before the refactoring. Notice that the web module contains a set of servlets.

The web module before microservices refactoring

The following figure shows the web project module after refactoring.

The web module after microservices refactoring

Refactor the existing implementations to use the microservice

The servlets in the web module call the TradeAction.java gateway dispatcher object in the dt-ejb EJB module to access core business logics. For example, getting market summary information requires the following web module codes:

TradeAction trade = new TradeAction();
MarketSummaryDataBean marketSummary = trade.getMarketSummary();

The TradeAction.getMarketSummary() method in turn invokes the dt-ejb TradeServices implementations to provide a business response. Earlier, when we refactored the dt-ejb module into new microservices, the public business methods in the TradeAction.java class, which the web module code depends on, became nonfunctional, breaking the existing web module code.

As a result, move the TradeAction.java class into a utility JAR project (the rest-client.jar file), and modify it to call the new microservices through the new RESTful APIs. The web module places this rest-client.jar file in its class path. This way, the existing web module codes can continue to use the TradeAction object without modifications as shown in the previous figure of the web module after microservices refactoring.

The following code shows how TradeAction queries the new marketsMicroService to obtain market summary information. The implementation pattern is the same for all other business operations that TradeAction supports. Because the web module code remains the same, you do not need to repeat it. The invokeEndpoint() method is a common utility method that you can create to simplify and reuse the RESTful API invocation codes.

       @Override
       public MarketSummaryDataBean getMarketSummary() throws Exception {
             String responseString = invokeEndpoint(marketsRoute + "/markets/", "GET");
             MarketSummaryDataBean responseObject =
mapper.readValue(responseString,MarketSummaryDataBean.class);
             return responseObject;
       }

Additionally, the TradeAction.java class that is used to reside in the dt-ejb module. It depends on the same set of modal objects and interfaces that you moved to the dt-ejb-client.jar file in Step 2b. Include this JAR file in the web module class path so that the TradeAction object can also use it.

Now that the TradeAction.java class is starting the new microservices via REST, it must be able to target them on different host names because the microservices can be hosted independently on a multitude of environments and networks. Therefore, add the correct parameters to the endpoint URL in the code so that an environment variable can retrieve it.

String accountsRoute = System.getenv("accounts_app_route");
    String marketsRoute = System.getenv("markets_app_route");
    String quotesRoute = System.getenv("quotes_app_route");
    String settingsRoute = System.getenv("settings_app_route");

Update the deployment and server configurations

Because you are moving the web module from the Daytrader3 enterprise application package and making it an independent microservice, you updated its pom.xml file to ensure that all dependencies and plug-ins are self-contained. The web module also requires a more simplified server.xml file if it is deployed into its own server container.

The following code listing shows the minimum required configurations.

       <server>
              <featureManager>
                      <feature>jaxrs-2.0</feature>
                      <feature>localConnector-1.0</feature>
              </featureManager>

              <httpEndpoint host="*" httpPort="9080" id="defaultHttpEndpoint"/>
           <webApplication id="web" location="web.war" name="web"/>
       </server>

The web tier must also specify the location of each of the microservices, which the server.env file handles.

accounts_app_route=http://localhost:9085/accounts
markets_app_route=http://localhost:9086/markets
quotes_app_route=http://localhost:9087/quotes
settings_app_route=http://localhost:9088/settings

Create integration tests

The integration tests for microservices are similar to the test that you did for the marketsMicroService in Step 2d. However, because each microservice is stand-alone, you must also add the capability to test them with their own stand-alone Liberty server. You must modify the pom.xml file to include the ability to download and start the Liberty server before you run the tests as you did in the EAR project.

Deploy and verify the microservices

You have now refactored all of the Daytrader submodules into their own respective microservices. You are ready to deploy and observe them working in collaboration. Currently, you have one web tier front end and four microservices, each of which can run in its own server. Deploying each module (that is the web tier and microservices) locally in separate Liberty servers requires a different HTTP port to prevent port conflicts. Having a different HTTP port might not be a concern if you are deploying them to separate hosts, IBM Cloud Public Cloud Foundry, or IBM Cloud Private.

Deployed and verified microservices

Start your application, and test it by entering the following URL:

http://localhost:9080/daytrader/

You can find the complete solution in the Part5Finish.zip file in the GitHub directory for this series.

Conclusion

You have now refactored an application to microservices. Each microservice is self-contained so that you can manage, scale, and deploy it independently in its own environment.

Acknowledgments

The authors thank Grzegorz Smolko and David Mulley for reviewing this part.